Forum Discussion
Automatic refreshes never pull in changes (UPDATED)
We incrementally refresh our semantic model every 30 minutes using the services's scheduler. As far as I know, scheduled refreshes perform a refresh type Automatic. This works fine.That makes little sense as the smallest partition is a daily partition. You will want to look into keeping the "today" partition in direct query mode.
Note: For your API refreshes you should replace "automatic" with "full" to force the refresh.
Hi markmsc,
Thank you for reaching out to the Microsoft Fabric Community Forum. Also, thanks to lbendlin, for his inputs on this thread.
The behaviour you’re seeing is expected with Automatic refreshes. Power BI only reprocesses partitions that aren’t already marked as Ready. If a partition is in the “Ready” state, the service assumes it’s up to date and skips it even if the source data has changed. That’s why scheduled refreshes work later, but on-demand automatic ones appear to do nothing.
Detect data changes: Make sure your incremental refresh policy uses a proper change-tracking column (e.g., ModifiedDate). This tells Power BI which partitions need refreshing. https://learn.microsoft.com/en-gb/power-bi/connect-data/incremental-refresh-overview#detect-data-changes
Query folding: Verify that filters on RangeStart and RangeEnd are folding to the source. If not, incremental logic may skip updates. https://learn.microsoft.com/en-gb/power-bi/connect-data/incremental-refresh-troubleshootPartition states: If partitions stay in “Ready,” Automatic refresh will skip them. You can confirm this via XMLA endpoint or Tabular Editor.
If you need data immediately, you can: Trigger a Full or Data Only refresh via the REST API instead of “Automatic.” Or update the change-tracking column in the source so the partition is re-evaluated. https://learn.microsoft.com/en-gb/rest/api/power-bi/datasets/refresh-datasetHope this clears it up. Let us know if you have any doubts regarding this. We will be happy to help.
Thank you for using the Microsoft Fabric Community Forum.
Thanks for your reply!
I'm a bit unclear on what you mean by your first sentence. What makes "little sense"?
We can't use DQ for the today partition for a couple of reasons. The incremental period for most of the tables in our model is five days, not just one ('today"). Also, we make extensive use of DAX functions in the model that are not available in DQ mode. It would require an extensive refactoring of the model to shift it to a hybrid model from import only.
I would prefer not to have to use Full, also for a couple of reasons. One, it is brute force, and refreshes data whether it is needed or not. Doing a Full refresh of all tables in the model might take longer on each pass than just running a schedueld incremental every 30 minutes, defeating the purpose. And to Full refresh only the partitions that have changed would be pretty burdensome, as I think, we'd have to essentially code the polling that the refresh already does to know which ones have changed.
From some more reading, I think the crux of the issue is that Automatic only does something if the partition is in a state other than Ready. All the partitions I am expecting to see changes in are just sitting there in Ready. I have no idea what makes a partition not Ready.
UPDATE: Perhaps part of my misunderstanding here is with how Full works. With Full on a specific partition, which is typically what I will do manually, I think it just refreshes the partition regardless of whether there are changes. Or if it doesn't, I never catch it doing the analysis step. But with Full on a table, I can see it doing the analysis step on each partition in the update window. So yes, perhaps Full is what I need to be doing in my code. I will change to that and test it.
I think the crux of the issue is that Automatic only does something if the partition is in a state other than Ready.Pretty much. "automatic" means "let the Power BI service decide if it wants to honor your refresh request".
Thanks for replying. This made me laugh. You seem like a properly jaded Power BI developer! 🙂
I agree with your sentiment entirely.