Scheduled Refresh using Dynamic Data Source

[n0ab]

Hello all-

A reporting project I'm currently working on uses a web hook to pull in ~50k records via AWS using a lambda function. As some of you may know, AWS has a forced 30 second timeout. This hinders the data pull in that we can only pull so many records at a time before we are force timed out. We've tried to get around this using a dynamic query that pulls in 10,000 records at a time into and then consolidates them into one data source, but once this source is published to the Power BI service, a scheduled refresh is not allowed due to the data source being dynamic. 

My question is this: is there a way to supplement this problem that avoids purchasing the report server capacities? I cannot seem to find a solution to my problem. I need to be able to pull in all records from this data source and place them on a scheduled refresh.

Thanks in advance for any assistance offered.

[v-xiaoyan-msft · Accepted solution]

Hi @n0ab ,

To refresh dynamic data sources in Power BI Service, you can use Web.Contents() and Relative Path parameter in it, please refer:

1. Setting a scheduled refresh on a Dynamic Data Source in Power BI 
2. Web.Contents(), M Functions And Dataset Refresh Errors In Power BI 
3. Using The RelativePath And Query Options With Web.Contents() In Power Query And Power BI M Code 

Hope it helps,

Community Support Team _ Caitlyn Yan

If this post helps then please consider Accept it as the solution to help the other members find it more quickly.

[Gabe_V]

Hi @v-xiaoyan-msft are there any similar solutions when using SharePoint.Tables? 

I am receiving the same error, but when using SharePoint.Tables instead of Web.Contents.

I have tried such things as putting Web.Contents inside of SharePoint.Tables and putting the root URL in a parameter and referencing that. Neither work.

Any assistance you could provide would be greatly appreciated.

Thanks!

[Anonymous]

Hi @all,

since a few month we´ve massive trouble with automated refresh of dynamic sources.

We used Web.Content with RelativePath before and it worked nice.

What happened? And how can I fix this issue?

I´ve checked all the links in the solution.
Many thanks in advance!

[Anonymous]

In addition the (well known) error message:

[MariaIsabel]

Hi Caitlyn,

I followed the instructions posted on the 3 links that you refered but I still have a problem with one of my sources: https://www.cuandoenelmundo.com/calendario/espana/2021. It doesn't have a query at the end but just change the url for every year. I wasn't able to use the Relativepath - query instructions with this one. Could anyone help to solve this please. 

I am trying to include a calendar with hollidays into my report that can be updated automatically.

Thanks in advance for your help

[mellon199]

Hi MariaIsabel

Were you able to find a solution for your query?