Automatic refresh of dynamic iterating web datasource

[Anonymous]

Hi.

I have a query in PBI Desktop that creates a list; 1..150, then iterates over a webpage and adds/expands the corresponding pages. I manually refresh the data in this pbix file every second day or so. Works like a charm.

I would like to somehow be able to set this up in the service for automatic daily refresh, but native service throws the error that the query is not supported, no news in that. BUT to my understanding it should actually be possible to create a dataflow based on the same query, and have that one refresh, but I am not even allowed to save the dataflow, as it says dynamic datasources are not supported. 

Surely many people must be facing these issues - what are you best workarounds? The solution must consider as few paid services as possible, as this is a hobby/demo-project only.

Also I once used a tool called Power Update, but cannot seem to find this anywhere, to automatically open, refresh, and close a pbix file. If anyone have a link to that tool, this could be an immense help and a fine solution aswell. I tried to generate a PS script to do this, but it ultimately c

Thanks.

[Anonymous]

Hi @Anonymous ,

Please try below ways:

1. Power Automate: Create a flow that triggers your Power Query script on a schedule and stores the result in a static data store which can then be accessed by Power BI.
2. You can consider using the combination of Windows Task Scheduler and PowerShell scripts to automate the refresh of a PBIX file.

         Here is a broad outline of the steps you would take:

• Write a PowerShell script that opens Power BI Desktop, refreshes the data, and publishes the report to Power BI Service.
• Schedule the PowerShell script using Windows Task Scheduler to run at your desired frequency.

Please note that automating tasks using scripts and Task Scheduler can be complex and may require troubleshooting to ensure reliability.

Best regards,
Community Support Team_Binbin Yu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi both,

Honestly I had hope for a much simpler solution to this rather simple task 😞

That said I have managed to schedule an attended flow that works (still struggling with the unattended one) in Power Automate. This required more steps both in PA and local system settings, than I can reproduce here, but hopefully it will be easier down the line, or maybe even the Desktop/Service query engines will be identical eventually...

[amustafa · Accepted solution]

Hi @Anonymous Perhaps the Power Automate Desktop can refresh the Power BI file for you. For PBI Service you can watch similar Youtube: Uncover the perfect domain name for your venture 15 (youtube.com)