Forum Discussion

n0ab's avatar
n0ab
New Member
4 years ago
Solved

Scheduled Refresh using Dynamic Data Source

Hello all-

 

A reporting project I'm currently working on uses a web hook to pull in ~50k records via AWS using a lambda function. As some of you may know, AWS has a forced 30 second timeout. This hinders the data pull in that we can only pull so many records at a time before we are force timed out. We've tried to get around this using a dynamic query that pulls in 10,000 records at a time into and then consolidates them into one data source, but once this source is published to the Power BI service, a scheduled refresh is not allowed due to the data source being dynamic. 

 

My question is this: is there a way to supplement this problem that avoids purchasing the report server capacities? I cannot seem to find a solution to my problem. I need to be able to pull in all records from this data source and place them on a scheduled refresh.

 

Thanks in advance for any assistance offered.

Need Help

6 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    4 years ago

    Hi n0ab ,

     

    To refresh dynamic data sources in Power BI Service, you can use Web.Contents() and Relative Path parameter in it, please refer:

    1. Setting a scheduled refresh on a Dynamic Data Source in Power BI 
    2. Web.Contents(), M Functions And Dataset Refresh Errors In Power BI 
    3. Using The RelativePath And Query Options With Web.Contents() In Power Query And Power BI M Code 

     


    Hope it helps,


    Community Support Team _ Caitlyn Yan


    If this post helps then please consider Accept it as the solution to help the other members find it more quickly.

    • MariaIsabel's avatar
      MariaIsabel
      New Member
      4 years ago

      Hi Caitlyn,

       

      I followed the instructions posted on the 3 links that you refered but I still have a problem with one of my sources: https://www.cuandoenelmundo.com/calendario/espana/2021. It doesn't have a query at the end but just change the url for every year. I wasn't able to use the Relativepath - query instructions with this one. Could anyone help to solve this please. 

      I am trying to include a calendar with hollidays into my report that can be updated automatically.

       

      Thanks in advance for your help

      • mellon199's avatar
        mellon199
        Frequent Visitor
        3 years ago

        Hi MariaIsabel

         

        Were you able to find a solution for your query? 

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi @all,

       

      since a few month we´ve massive trouble with automated refresh of dynamic sources.

      We used Web.Content with RelativePath before and it worked nice.

      What happened? And how can I fix this issue?

      I´ve checked all the links in the solution.
      Many thanks in advance!

       

      • Anonymous's avatar
        Anonymous
        Not applicable
        2 years ago

        In addition the (well known) error message:

         

    • Gabe_V's avatar
      Gabe_V
      Helper I
      1 year ago

      Hi Anonymous are there any similar solutions when using SharePoint.Tables? 

      I am receiving the same error, but when using SharePoint.Tables instead of Web.Contents.

      I have tried such things as putting Web.Contents inside of SharePoint.Tables and putting the root URL in a parameter and referencing that. Neither work.

       

      Any assistance you could provide would be greatly appreciated.

      Thanks!