Web Data Source Limitations?

[bcharlto]

Are there data size limitations when connecting to a web data source?

My data source only has 560K rows and was originally loading, however, I am now getting a "DataSourceError: The operation has timed out". The data source is working for my colleagues in the US while I am in a Canadian office.

I have reinstalled Power BI, deleted temporary files, created new files, and am still getting issues with one specific URL.

Thank you for your time and help!

[Anonymous]

@bcharlto,

Please, go to query editor, right click your data source and click Advanced Editor, set proper Timeout value in the codes of Advanced Editor, then check if you can connect to the web source in Power BI Desktop.

let    
   Source = Web.Contents ("http://URL ", [Timeout=#duration(0, 30, 0, 0)])
in    
   Source

There is a similar thread for your reference.

https://social.technet.microsoft.com/Forums/en-US/d54165e2-04e4-44e3-888b-792dbe624660/power-query-h...

Regards,
Lydia

[bcharlto]

@Anonymous,

Thank you for the reply. I have also read through the link and done some more research, however my error now reads: "DataFormat.Error: We reached the end of the buffer."

It did not run 30 minutes before getting this error.

let
    Source = Json.Document(Web.Contents("https://URL", [Timeout=#duration(0,30,0,0)])) in Source

I would try this in a new file, however I can not load the query in order to access the Advanced Editor settings.

Is there something related to this being an https link?

[Anonymous]

@bcharlto,

You can add a new blank query, then paste the code in the Advanced Editor of the blank query. Replace the URL with your own URL. In addition, is there any possibility that you get limited data by adding required parameter in  the URL?

let
    Source = Json.Document(Web.Contents("URL", [Timeout=#duration(0,30,0,0)])) 
in 
    Source

Regards,
Lydia

[bcharlto]

@Anonymous,

Thank you for showing how to load into a blank query. This part worked. It loaded (588K Rows), I converted it to a table, expanded the 14 columns and all was well... until I hit refresh and got an error.

Refreshed in Query Editor:

DataSource.Error: Web.Contents failed to get contents from 'URL' (500):

Details:

DataSourceKind=Web

DataSourcePath=URL

URL=URL

I backed out the column expansion and the conversion to a table and it refreshed properly again. I then converted back to a table, expanded columns, closed query editor, and hit refresh, getting the error below again:

DataSource.Error: Web.Contents failed to get contents from 'URL' (502): Bad Gateway

What do you think? I still can't believe my colleagues in the US are not having this issue...

[Anonymous]

@bcharlto,

The above error could occur when you return too much data from the data source. I would recommend you import the reuqired rows and columns in Power BI Desktop. 

There is also a similar blog for your reference.
http://blogs.catapultsystems.com/cfuller/archive/2017/09/07/how-to-configure-power-bi-to-regularly-u...

Regards,
Lydia