Allow Detect Data Changes for Historic Data Outside Incremental Window
In Power BI Incremental Refresh,Detect Data Changes only works inside the incremental refresh window. If historic data gets updated, Power BI cannot detect or refresh it unless I expand the incremental window.
Although we can refresh historic partitions manually through XMLA/SSMS, but it would be great if this capability built directly in power BI.
Please add an option to detect changes and refresh historic partitions even when they are outside the incremental window. This will help with scenarios where old records are updated occasionally.
Thanks in advance!
Keerthi
1 Comment
- messias93New Member
Hi Keerthi,
I ran into this exact issue and found a workaround that solved it completely.
The fix: set the "Incrementally refresh data starting" window to the same duration and granularity as your archive window ("Store rows in the last"). For example, if you store 10 years of history, set the incremental window to 10 years too — instead of a small window like 30 days.
Detect Data Changes only checks partitions that fall inside the incremental window. If that window is small, historic partitions are simply never checked. Making the window match your full archive means every partition gets checked on every refresh — and thanks to the refresh bookmark, unchanged partitions are still skipped automatically, so you're not paying for a full reload each time, just a lightweight check per partition.
One key adjustment to make this practical: reduce your partition count. With a wide incremental window, every refresh checks every partition, so more partitions means more overhead. Switching from monthly to yearly granularity cut our total partition count by ~10x and dramatically reduced refresh duration.
With this approach, we went from a full refresh to fully automatic incremental detection across a 10-year history, and measured a ~70-76% reduction in Fabric CU consumption compared to the original setup — validated across multiple days, not a one-off measurement.
Happy to share more detail if useful.
Recent ideas
Support Fabric SQL Database with workspace-level inbound Private Link
Fabric SQL Database supports tenant-level Private Link but not workspace-level Private Link. Securing a small number of databases therefore requires enabling Private Link across the entire tenant, in...WorkFull2214 minutes agoAdvocate INew23Views1like1CommentPaste values in slicers for hierarchy data
Hierarchical slicers are widely used in enterprise reports (for example Product RCN > SKU Key, Region > Country > City, or GPP > Division > RCN > SKU). Users often need to bulk-filter by pasting valu...eddieisking20001 hour agoFrequent VisitorNew4Views0likes0CommentsPower Query Parameter create/use when connecting to a dataflow
In power query when you connect to a table in a dataflow there are no options to create or use a parameter for that dataflow. When connecting to SQL there is an option when setting up the connection ...ben_holmes4 hours agoNew MemberNeed Clarification204Views3likes2CommentsMake Copilot Approval status for semantic model available through API
In recent versions fabric allowed to mark a semantic model as approved for copilot to reduce AI adoption friction. However there is no specific way for external governance application to analyse whi...FrançoisD4 hours agoFrequent VisitorNew12Views2likes0CommentsClone a workspace, including its data, to a new workspace
Description: Today there's no way to fully clone a Fabric workspace: git sync and Deployment Pipelines move item definitions only, not the underlying data; Warehouse's zero-copy CREATE TABLE ... AS ...gclements5 hours agoHelper IINew14Views1like0Comments