Forum Discussion
SQL Fabrics – the report doesn’t get the updated data from SQL Fabrics
Hi everyone,
I have a very simple setup in fabrics, a dataflow gen 2 for ETL that saves the data in a different workspace in a SQL fabrics DB. The tables are preconfigured, and the data is only replaced by the dataflow.
The dataflow updates at 5 AM (all successful) and the report that relies on the SQL Fabrics DB at 5.30 AM. However, I’ve noticed that the updated data is not reflected in the report. When I manually update the report (usually around 8-9 AM), I get the right data.
Did anyone else encounter this problem? It looks like the report is getting data from a cache or something rather than the DB itself (maybe there are some settings that I need to change or something).
LE: My current configuration is: Dataflow Gen2 (Workspace 1 - Fabric) -> SQL Fabrics DB (Workspace 2 - Fabric) -> Semantic Model (Workspace 3 – Pro). The problem is the Semantic model that doesn’t pick up the latest data. The semnatic model is scheduled to run the update 30 min after the DB is updated (the DB is updated in less then 2 min after the dataflow is tirggered)
10 Replies
Hi,
could you please clarify how you´re updating the report?
Are you using a refresh activity for the semantic model at the end of your dataflow/pipeline, and then doing a manual refresh of the semantic model around 8-9 AM?Hi pagen ,
Thank you for reaching out to the Microsoft Community Forum.
Please try below things to fix the issue.
1. Disable Query Caching in Power BI, Go to Power BI Service --> Dataset Settings, Under Performance Options, set Query Caching to Off.
2. Add a wait time like 5–10 minutes after the Dataflow completes before triggering the report refresh. Alternatively, use a notebook or script to verify that the SQL endpoint is synced before refreshing the report.
3. If your data contains DateTime column, configure Incremental Refresh in Dataflow Gen2. This reduces load and ensures only new data is processed.
4. Manually refresh the report as you have been doing around 8–9 AM ensures fresh data is loaded.
Note: Check the refresh history in both Dataflow and Power BI to identify any timing mismatches. If possible use Direct Lake instead of SQL endpoint for faster and more reliable data access.
Please refer below links.
Solved: Re: DataFlow Gen2 runs successfully but data is mi... - Microsoft Fabric Community
Incremental refresh in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
Dataflow Gen2 refresh - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
Hi v-dineshya ,
1: The report is published in a Power BI Pro workspace, and there is no Query Caching option available in Pro workspace (see the SS comparison between Pro and PPU).
My current configuration is: Dataflow Gen2 (Workspace 1 - Fabric) -> SQL Fabrics DB (Workspace 2 - Fabric) -> Semantic Model (Workspace 3 – Pro). The problem is the Semantic model that doesn’t pick up the latest data.
2: as I mentioned, there is a 30 min gap between the semantic refresh and SQL update (I'm going to test different time gaps to see if there is a "golden" time lag)
3 & 4: I don’t see how these apply to my problem.
Hi pagen and v-dineshya,
I had a similar issue and tried steps 1 and 2 (as v-dineshya suggested), but they didn’t help in my case.
I have a pipeline that loads the data from bronze to gold, and at the end there’s a refresh activity that’s supposed to update the semantic model.As far as I know, the refresh activity performs an enhanced refresh, which doesn’t clear the cache but only updates the data.
With a scheduled refresh directly in the semantic model, or a manual refresh, both the cache is cleared and the data is updated. You can see this difference in the refresh history of the semantic model.In Data Factory, the refresh only updates the data, whereas a scheduled refresh updates both the data and the query cache.
Technically, you’d probably need to trigger the query cache invalidation via a REST API call at the end of the pipeline — but for me, that feels too complicated for this problem.
My solution was to set up a scheduled refresh about 30 minutes after the pipeline finishes.
I’m not sure if there’s a better solution, but this was the only thing that worked for me.
Edited: Of course the refresh button in the report or app did not work either. 😉
Hi pagen ,
Power BI Pro Workspace does not include the Query Caching setting, but Premium Per User (PPU) Workspace include the Query Caching setting, your semantic model is in a Pro workspace, you cannot disable Query Caching, which might be causing the delay in reflecting updated data from the SQL Fabric DB.
Please try below workarounds to fix the issue.
1. If possible, publish the semantic model to a PPU workspace where you can disable Query Caching.
2. Instead of querying via the SQL endpoint, use Direct Lake mode for faster and more consistent data access. This avoids caching issues and checks near real-time data availability.
3. Please try to increase the delay between the SQL DB update and the semantic model refresh. even though the DB update finishes quickly, the data might not be fully committed or indexed.
4. Before triggering the semantic model refresh, use a Fabric notebook or a small script to query the SQL endpoint and confirm that the data is updated.
5. If you are using Fabric pipelines, you can set up dependent activities so that the semantic model refresh only triggers after the SQL DB confirms the update.
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
