Forum Discussion
Impactful IndividualIncremental Refresh Decision
Since it is a new semantic model, I would recommend deciding the incremental-refresh one before production. If the fact table is large and growing and if the source supports query folding then configuring incremental refresh in Dev or Test and validating the refresh and then deploying the tested model to Production.
Can you confirm these
What is the current and expected data volume/row count
How much historical data needs to be retained
How frequently is the data refreshed
What is the source system
Is query folding supported for the date/time filter
Is the data append-only or can historical records be updated/deleted
Impactful IndividualHi Natarajan_M , krishnakanth240
how i should handle the historical data changes in incremental refresh? also the current size of dataset is 1 M however in future it may grow so should i implement incremental at this stage?
Super UserSince historical records can be updated or deleted, no need to rely on 'Refresh rows in the last N day setting' that only catches changes within refresh window and will miss edits to earlier data.
You can add 'Detect data changes' using LastModifiedDate or ModifiedTimestamp column from data source which makes Power BI check which rows changed even outside the normal refresh range and only reprocess those partitions
Should you implement it now at 1M rows? Yes you can implement it. Row count alone is not deciding factor, it's 5 year requirement.
Incremental refresh is easier to design correctly from start (partition ranges, RangeStart/RangeEnd parameters, policy settings) than to fit later post model is in production and reports depend on it
Configure and validate it fully in Dev/Test then confirm detect changes column works as expected