Forum Discussion
Does incremental refresh cascade down to referenced/merged queries?
- 5 months ago
Incremental refresh does not cascade. You need to configure it separately on AuditWF, AUFI_1, and AUFI_2.
One thing to flag: SharePoint Excel files don't support query folding, so incremental refresh won't work as expected. It will pull the full data every time. Move to a SQL or Lakehouse source if you want incremental refresh to actually work. - 5 months ago
Hi olimilo,
Thank you for posting your query in the Microsoft Fabric Community Forum.Incremental refresh policies are configured at the individual table/query level and do not automatically propagate to referenced or merged queries. Therefore, you do not need to duplicate AuditWF, but if AUFI_1 and AUFI_2 are also loaded as separate tables and require incremental refresh behavior, you'll need to configure an incremental refresh policy for each of those tables separately.
Regarding your question about using SharePoint file metadata such as Modified Date, filtering based on metadata can help reduce the amount of data processed, depending on how the query is evaluated. However, incremental refresh relies on query folding to push filters to the data source. If the source does not support query folding, filtering might not be applied at the source level, which can reduce the effectiveness of incremental refresh.
In general, incremental refresh delivers the best performance when query folding is available, allowing filters to be applied at the source.
For more information, please refer to the official documentation:
Using incremental refresh with dataflows - Power Query | Microsoft Learn
Incremental refresh in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
Thanks, Kedar_Pande for sharing valuable insights.
Best regards,
Ganesh Singamshetty
Incremental refresh does not cascade. You need to configure it separately on AuditWF, AUFI_1, and AUFI_2.
One thing to flag: SharePoint Excel files don't support query folding, so incremental refresh won't work as expected. It will pull the full data every time. Move to a SQL or Lakehouse source if you want incremental refresh to actually work.
> You need to configure it separately on AuditWF, AUFI_1, and AUFI_2.
So in essence, I would have to duplicate the AuditWF and apply the inc refresh rules for each instance?
> SharePoint Excel files don't support query folding, so incremental refresh won't work as expected
Is it not possible to check the files using the Modified Date? Similar to what was demonstrated here:
Faster SharePoint folder consolidation using Incremental Refresh (see warning in the notes)