Forum Discussion

bcharlto's avatar
bcharlto
Frequent Visitor
8 years ago

Web Data Source Limitations?

Are there data size limitations when connecting to a web data source?   My data source only has 560K rows and was originally loading, however, I am now getting a "DataSourceError: The operation has...
Need Help
Anonymous's avatar
Anonymous
Not applicable
8 years ago

bcharlto,

Please, go to query editor, right click your data source and click Advanced Editor, set proper Timeout value in the codes of Advanced Editor, then check if you can connect to the web source in Power BI Desktop.

let    
   Source = Web.Contents ("http://URL ", [Timeout=#duration(0, 30, 0, 0)])
in    
   Source


There is a similar thread for your reference.

https://social.technet.microsoft.com/Forums/en-US/d54165e2-04e4-44e3-888b-792dbe624660/power-query-http-timeout-setting?forum=powerquery

Regards,
Lydia

bcharlto's avatar
bcharlto
Frequent Visitor
8 years ago

Anonymous,

 

Thank you for the reply. I have also read through the link and done some more research, however my error now reads: "DataFormat.Error: We reached the end of the buffer."

 

It did not run 30 minutes before getting this error.

 

let
    Source = Json.Document(Web.Contents("https://URL", [Timeout=#duration(0,30,0,0)])) in Source

 

I would try this in a new file, however I can not load the query in order to access the Advanced Editor settings.

 

Is there something related to this being an https link?

  • Anonymous's avatar
    Anonymous
    Not applicable
    8 years ago

    bcharlto,

    You can add a new blank query, then paste the code in the Advanced Editor of the blank query. Replace the URL with your own URL. In addition, is there any possibility that you get limited data by adding required parameter in  the URL?


    let
    Source = Json.Document(Web.Contents("URL", [Timeout=#duration(0,30,0,0)])) 
in 
    Source




    Regards,
    Lydia

    • bcharlto's avatar
      bcharlto
      Frequent Visitor
      8 years ago

      Anonymous,

       

      Thank you for showing how to load into a blank query. This part worked. It loaded (588K Rows), I converted it to a table, expanded the 14 columns and all was well... until I hit refresh and got an error.

       

      Refreshed in Query Editor:

      DataSource.Error: Web.Contents failed to get contents from 'URL' (500):

      Details:

      DataSourceKind=Web

      DataSourcePath=URL

      URL=URL

       

      I backed out the column expansion and the conversion to a table and it refreshed properly again. I then converted back to a table, expanded columns, closed query editor, and hit refresh, getting the error below again:

       

      DataSource.Error: Web.Contents failed to get contents from 'URL' (502): Bad Gateway

       

      What do you think? I still can't believe my colleagues in the US are not having this issue...