Forum Discussion
Impactful IndividualIncremental Refresh Decision
I am creating fresh (new) semantic model, I am not sure whether I should implement incremental refresh now or later point of time . I do not want to change or overwrite semantic model once it move to production envionment since it may impact existing reports. ?
3 Replies
powerbiexpert22What is the current and expected data volume/row count: current -1 million rows, expected 100-500 rows per day
How much historical data needs to be retained : last 5 years
How frequently is the data refreshed: Daily
What is the source system: Databricks
Is query folding supported for the date/time filter: Yes
Is the data append-only or can historical records be updated/deleted: historical can be updated/deleted
- krishnakanth240
Super User
Since historical records can change, you can configure Incremental Refresh and Detect data changes instead seeing the table as append only.
You can validate it in Dev/Test first then deploy tested semantic model to Production which avoids having to introduce the refresh architecture as model is already in production
Given the low daily growth, incremental refresh is useful because of 5-year historical window and potential updates/deletes to historical data.
- krishnakanth240
Since it is a new semantic model, I would recommend deciding the incremental-refresh one before production. If the fact table is large and growing and if the source supports query folding then configuring incremental refresh in Dev or Test and validating the refresh and then deploying the tested model to Production.
Can you confirm these
What is the current and expected data volume/row count
How much historical data needs to be retained
How frequently is the data refreshed
What is the source system
Is query folding supported for the date/time filter
Is the data append-only or can historical records be updated/deleted
