Forum Discussion
Impactful IndividualIncremental Refresh Decision
Since it is a new semantic model, I would recommend deciding the incremental-refresh one before production. If the fact table is large and growing and if the source supports query folding then configuring incremental refresh in Dev or Test and validating the refresh and then deploying the tested model to Production.
Can you confirm these
What is the current and expected data volume/row count
How much historical data needs to be retained
How frequently is the data refreshed
What is the source system
Is query folding supported for the date/time filter
Is the data append-only or can historical records be updated/deleted
Super UserHi powerbiexpert22 ,
Good to think this through before prod. Given your numbers:
- Adding incremental refresh later won't break anything. It doesn't touch your model/measures — the engine just re-partitions and does a one-time full reload (a few mins). Only caveats: you can't re-download the .pbix after, and republishing from Desktop can reset partitions. Safe to add later, just do it deliberately.
- At your volume it's overkill. Incremental earns its keep at tens of millions of rows / several million per month. You're at ~1M total — a small model full refresh handles easily.
- Refresh time isn't your problem. Databricks → Power BI, ~1M rows folds in about 2–5 mins. Nothing to optimize.
- Your mutable history actually argues against it. Incremental only re-queries the recent window and freezes older partitions. Since your records can be updated/deleted across all 5 years, those edits sit in archived partitions incremental won't refresh — so you'd show stale data. That's a correctness risk. Full refresh always reflects updates/deletes.
- If you want proof, pilot both. One full, one incremental in QA for a week with real historical edits — compare refresh time and accuracy before deciding.
Bottom line: start with full refresh — simpler, fast enough, and the only option that stays accurate with editable history. Revisit incremental only if volume grows into the millions and history becomes append-only. Adding it later costs you nothing.
Thanks
Natarajan Manivasagan
If you found this helpful, please consider giving it a Kudos and marking it as the accepted solution — it goes a long way in helping others facing the same issue.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
🏆Best Solution for Enterprise BI — 2026 Microsoft Fabric Semantic Link Developer Experience Challenge
👉Microsoft announcement · View the winning notebook
For more Power BI tips and discussions, let's connect on LinkedIn.
Cheers!
powerbiexpert22Hi Natarajan_M , krishnakanth240
how i should handle the historical data changes in incremental refresh? also the current size of dataset is 1 M however in future it may grow so should i implement incremental at this stage?
- krishnakanth240
Since historical records can be updated or deleted, no need to rely on 'Refresh rows in the last N day setting' that only catches changes within refresh window and will miss edits to earlier data.
You can add 'Detect data changes' using LastModifiedDate or ModifiedTimestamp column from data source which makes Power BI check which rows changed even outside the normal refresh range and only reprocess those partitions
Should you implement it now at 1M rows? Yes you can implement it. Row count alone is not deciding factor, it's 5 year requirement.
Incremental refresh is easier to design correctly from start (partition ranges, RangeStart/RangeEnd parameters, policy settings) than to fit later post model is in production and reports depend on it
Configure and validate it fully in Dev/Test then confirm detect changes column works as expected
- Natarajan_M
Hi powerbiexpert22 ,
Could you please share the current refresh times for the semantic model in the following scenarios?- Average time required for a full refresh.
- Average time required for an incremental refresh, including the "detect data changes" feature.
One potential pitfall of using "detect data changes" is that it triggers a pre-polling activity, which can sometimes increase the total refresh time. Ideally, 1 million rows is negligible compared to a 50 million row model in import mode which most semantic model will be.
Additionally, what are the current model size and the capacity size assigned to it?
please see the below blog for more details :https://www.linkedin.com/pulse/power-bi-incremental-refresh-detect-data-changes-sarbjit-singh-n5ltc/
Thanks- powerbiexpert22
Hi Natarajan_M , i think it will take some time for capturing the information, i will create new thread if needed , thanks