Forum Discussion
Helper IIncremental refresh is not working as expected
I have setup incremental refresh in powerbi using one of the date/time column and published to service. When i go and refresh it for the 2nd time also, it is taking same amount of time. My data source is Snowflake.
1.
2. = Table.SelectRows(CASES_Table, each [LASTMODIFIEDDATE] >= RangeStart and [LASTMODIFIEDDATE] < RangeEnd).
3.
Hi Mahipal,
One thing I would add is that the second refresh taking roughly the same amount of time does not necessarily mean incremental refresh is not working.
With your current policy, Power BI will refresh the entire last 5 days on every service refresh. Microsoft's incremental refresh documentation states that all rows within the configured refresh period are refreshed each time.
In your screenshot, Detect data changes is not enabled, so Power BI is not trying to determine whether individual daily partitions actually changed before refreshing them.
The first refresh is different because it creates and loads the historical partitions as well as the recent refresh partitions. After that, only the configured refresh window should normally be processed.
So I would verify what Snowflake is actually receiving before concluding that incremental refresh is failing. In Snowflake Query History, compare the SQL generated by the first and second Power BI Service refreshes and check whether the later refresh contains LASTMODIFIEDDATE predicates covering only the recent 5-day window.
If the second refresh is querying only those recent dates, then incremental refresh is working even if the total duration happens to be similar.
If you have another reliable audit/change timestamp, you could also consider Detect data changes. Microsoft documents that this can skip periods whose maximum change value has not changed since the previous refresh. I would not use the same column that you are already using for RangeStart / RangeEnd partitioning.
So I would check:
- whether the RangeStart / RangeEnd filter folds to Snowflake;
- what date predicates appear in Snowflake Query History on the second refresh;
- how much of your total data volume actually falls inside those five days.
That should tell you whether the issue is the incremental-refresh policy or simply that the five-day refresh window itself is still expensive.AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
4 Replies
v-sathmakuriCommunity Support
Hi Mahipal ,
Could you review the suggestion provided above and let us know if you have any additional questions, we are happy to address.
Thanks!!
- Kannan_n
Advocate III
Hi Mahipal,
To pinpoint why the second refresh isn't faster, check these three specific areas in addition to query folding:
- Snowflake Warehouse State (Cold vs. Warm Cache): If your Snowflake virtual warehouse auto-suspends between refreshes, the second run hits cold local SSD cache and has to spin up compute from scratch, often taking similar time even for fewer rows.
- Clustering & Micro-partition Pruning: Check the Query Profile in Snowflake for the second refresh. If LASTMODIFIEDDATE isn’t part of the clustering key, Snowflake may perform a full table micro-partition scan (partitions scanned equals partitions total) despite the WHERE clause.
- Custom SQL Import: If the query was loaded using custom SQL (Value.NativeQuery) rather than navigating through schema tables, incremental partition filters break folding unless [EnableFolding=true] is explicitly added in M.
If this helped resolve your issue, please mark it as the Accepted Solution and drop a kudos to help others in the community find it!
Thanks
Kannan N
- ShivekMaharaj
Memorable Member
Hi Mahipal,
One thing I would add is that the second refresh taking roughly the same amount of time does not necessarily mean incremental refresh is not working.
With your current policy, Power BI will refresh the entire last 5 days on every service refresh. Microsoft's incremental refresh documentation states that all rows within the configured refresh period are refreshed each time.
In your screenshot, Detect data changes is not enabled, so Power BI is not trying to determine whether individual daily partitions actually changed before refreshing them.
The first refresh is different because it creates and loads the historical partitions as well as the recent refresh partitions. After that, only the configured refresh window should normally be processed.
So I would verify what Snowflake is actually receiving before concluding that incremental refresh is failing. In Snowflake Query History, compare the SQL generated by the first and second Power BI Service refreshes and check whether the later refresh contains LASTMODIFIEDDATE predicates covering only the recent 5-day window.
If the second refresh is querying only those recent dates, then incremental refresh is working even if the total duration happens to be similar.
If you have another reliable audit/change timestamp, you could also consider Detect data changes. Microsoft documents that this can skip periods whose maximum change value has not changed since the previous refresh. I would not use the same column that you are already using for RangeStart / RangeEnd partitioning.
So I would check:
- whether the RangeStart / RangeEnd filter folds to Snowflake;
- what date predicates appear in Snowflake Query History on the second refresh;
- how much of your total data volume actually falls inside those five days.
That should tell you whether the issue is the incremental-refresh policy or simply that the five-day refresh window itself is still expensive.AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
- grazitti_sapna
Super User
Hi Mahipal,
Doesn't seem any issue with your incremental refresh setup.
Please check below points:-
- Confirm the column data type (Should be date time)
- Check the position of the filter step, should be immediatly after the Snowflake table is selected and befor any tranformations, e.g.
let Source = Snowflake.Databases(...), Database = Source{[Name="YourDatabase"]}[Data], Schema = Database{[Name="YourSchema"]}[Data], CASES_Table = Schema{[Name="CASES"]}[Data], FilteredCases = Table.SelectRows( CASES_Table, each [LASTMODIFIEDDATE] >= RangeStart and [LASTMODIFIEDDATE] < RangeEnd ) in FilteredCases - Test query folding - Check for View native query and make sure that the SQL contains the date filter and is being passed to snowflake correctly
WHERE LASTMODIFIEDDATE >= ... AND LASTMODIFIEDDATE < ... - Check Snowflake query history - In Snowflake, inspect the SQL generated during the Power BI refresh. it should contain:-
WHERE LASTMODIFIEDDATE >= '2026-09-07' AND LASTMODIFIEDDATE < '2026-09-12' - Check whether the first service refresh completed successfully - The first refresh after publishing creates the historical and incremental partitions. If the initial refresh failed, was cancelled, or the model was republished afterward, the expected partition structure may not exist. Refer documentation
- Microsoft recommends that the change-detection column generally be different from the column used to partition the data.
- Check date format in Power BI and snowflake
🌟 I hope this solution helps you unlock your Power BI potential! If you found it helpful, click 'Mark as Solution' to guide others toward the answers they need.
💡 Love the effort? Drop the kudos! Your appreciation fuels community spirit and innovation.
🎖 As a proud SuperUserand Microsoft Partner, we’re here to empower your data journey and the Power BI Community at large.
🔗 Curious to explore more? [Discover here].
Let’s keep building smarter solutions together!
