Forum Discussion
Dataflow Incremental Refresh Detect Data Changes
Hi brandonyeo Power BI Gen1 Dataflows don’t let you modify the Canary function used for detecting changes. It always uses the full query, so performance gains are limited in complex cases. To optimize, create a lightweight SQL view or staging dataflow with just primary key and MODIFIED\_DATE. You can also consider using Gen2 Dataflows for better incremental refresh control.
Hi brandonyeo Power BI Gen1 Dataflows don’t let you modify the Canary function used for detecting changes. It always uses the full query, so performance gains are limited in complex cases. To optimize, create a lightweight SQL view or staging dataflow with just primary key and MODIFIED\_DATE. You can also consider using Gen2 Dataflows for better incremental refresh control.
Hi Akash_Varuna, thanks for your reply. Yes, I ended up doing something like this - Add another lightweight view that gets the P_KEY and MODIFIED_DATE without the incremental refresh filter, then merging it with the original query as an inner join. Thankfully all historical changes in the db are stored in a separate audit table so the main table has unique P_KEY values only.
Unfortunately, we have not migrated entirely onto Fabric, so Gen2 Dataflows is not available for us.