Forum Discussion
Help needed: Detect Data Changes on Modified column with Incremental Refresh
Hi Natarajan_M ,
I've implemented a last 60 months incremental refresh with data detect changes on my dataset. However, with Power BI Premium Per User (PPU) license, the dataset is not refreshing completely and I'm encountering timeout errors.
Could you please help me resolve this issue?
Thanks!
Hi automation,
This behavior is expected when using a large incremental refresh window with Detect data changes on a Power BI Premium Per User (PPU) license. With a last 60 months refresh policy enabled, the service must evaluate every partition within that window during each refresh to check for changes. Even though Detect data changes helps avoid unnecessary reloads, Power BI still issues polling queries per partition, and if many partitions are involved, this can quickly exceed the execution and resource limits of PPU, resulting in refresh timeouts.
It’s also important to note that Detect data changes operates at the partition level, not at the individual row level. If a single row changes within a large partition, the entire partition needs to be reprocessed, which further increases refresh cost and duration. This becomes especially noticeable when partitions span long periods such as months or years.
To mitigate this, you can try reducing the incremental refresh window (for example, from 60 months to a smaller range that aligns with how often historical data actually changes), and ensure that the column used for RangeStart/RangeEnd and Detect data changes supports full query folding back to the source. Proper partitioning and folding are critical to keeping refreshes efficient.
Thanks,
prashanth