Forum Discussion
Pro Incremental Refresh with Detect Data Changes fails
I managed to solve this by introducing another column for detect changes. Microsoft documentation is also now updated with this change.
Now I have two columsn:
1. Record timestamp that I use for incremental refresh and in power query.
2. Record modification timestamp that I use in setting up the detect changes.
Advanced policy options
Detect data changes
Incremental refresh of ten days is more efficient than full refresh of five years. However, it's possible to do even better. If you select the Detect data changes checkbox, you can select a date/time column used to identify and refresh only the days where the data has changed. This assumes such a column exists in the source system, which is typically for auditing purposes. This should not be the same column used to partition the data with the RangeStart/RangeEnd parameters. The maximum value of this column is evaluated for each of the periods in the incremental range. If it has not changed since the last refresh, there is no need to refresh the period. In the example, this could further reduce the days incrementally refreshed from ten to around two.