Dataset not updating

[ClaraVL]

I have the following set-up:

• Power BI PPU Workspace A: several chained DataFlows that extract (and load) and combine/transform data (using linked tables)
• Power BI PPU Workspace B: Report with Dataset which connects to the final dataflow in Workspace A
• Power Automate Flow: Automated trigger on Dataflow completion which triggers dataset updates

In Workspace A I have an hourly refresh schedule which works as expected. I see all updates when I open the dataflow and preview the data. The specific query I'm interested in uses a computed table. The Flow also works as expected.

In Workspace B, however - despite both the dataflow and dataset succesfully refreshing - the data does not get updated with the latest data. I've checked by refreshing the report and connecting from Excel. I've also manually refreshed the dataset a few more times without success.

I do notice that dataset eventually updates, so far this seems to happen maximum a day later.

There is no connected Azure Data Lake Storage account, and I'm not using M parameters.

Any thoughts on why the data does not update?

--- Update
When refreshing the report in Power BI Desktop I also see outdated data.

[ClaraVL]

Thanks for the response @JoeBarry!

The source of the extract dataflow (outside Power BI), updates continuously. This extract dataflow takes a snapshot every hour. Then the chained transform dataflow automatically refresh as well. This takes about 1 minute. When I look at the data in this transform dataflow, everything is as expected. Both have load enabled.

It's only at the next stage, the dataset in workspace B, where the data is outdated. So I have both actual data (workspace A) and outdated data (workspace B). All of this happens within the Power BI Service, same tentant, but across workspaces. All dataflows and datasets show they're refreshed.

[JoeBarry]

I would say there is an issue with the PA Flow. Maybe check that to see how long it takes to run? If the data is appearing later then it could be related to this.

Maybe there is a Refresh of the dataset running and it can't complete it's task until it finishes. Is there also a scheduled refresh of the dataset setup?

[ClaraVL · Accepted solution]

I've rebuilt the entire workspace, by exporting and importing the dataflows and updating the tables sources.

This seems to have solved the issue, will monitor for some time to be sure. 

Very strange...

[JoeBarry]

Hi @ClaraVL 

Very strange indeed, glad you've found a solution that works for you.

all the best 

Joe

[ClaraVL]

Still working fine, so rebuilding the source workspace has resolved it. Thanks for the prompt responses @JoeBarry !

[ClaraVL]

The Power Automate Flow takes 7 seconds. It starts after completion of the DataFlow refresh, and refreshes the dataset without issues. There is no scheduled refresh set on the dataset, so there should not be any parallel refreshes taking place.

I've also succesfully manually refreshed the dataset a few times afterwards indeed to rule out it was caused by the PA Flow. Same result.

[JoeBarry]

Hi @ClaraVL 

How often does the source of the Dataflows update? If you know this, schedule the dataflow refresh for after this time and the Report refresh after the dataflow refresh. Look at the average refresh time and give your refreshes that average +30 minutes in between

Thanks

Joe

If this post helps, then please Accept it as the solution