incremental refresh

Hi,

Incremental refresh can definitely help reduce the refresh time and manage a ~3 GB Import semantic model, but the fact that source data can be deleted is an important consideration.

By default, incremental refresh only refreshes the period defined in the incremental refresh policy. Therefore, if a record from an older/historical partition is deleted at the source, Power BI may not detect that deletion because that historical partition is no longer being refreshed.

Recommended approach

I would configure incremental refresh with a refresh window that covers the period in which updates/deletions can occur.

For example, if records can be modified or deleted within the last 30 days:

• Store/archive: 5 years
• Incrementally refresh: last 30 days
• The last 30 days will be reloaded on every refresh.
• Older partitions will remain unchanged.

This means a deletion that occurs within those 30 days will be reflected in Power BI during the next refresh.

If deletions can happen at any time, you need to consider a different approach, such as:

1. Increase the incremental refresh window so that it covers the maximum period in which historical records can be deleted.
2. If the source provides an audit/change tracking column, use the Detect data changes option where appropriate. Microsoft documents this as a way to identify periods where data has changed.
3. If historical records can be deleted without any reliable way to identify those deletions, incremental refresh alone cannot guarantee that Power BI will discover them in old partitions. In that situation, consider periodically refreshing the historical partitions/full model or implementing a proper change-data-capture/ETL process upstream.

Important note about the first refresh

One thing to keep in mind is that incremental refresh does not immediately make the first service refresh fast.

After publishing the model, the initial refresh creates the historical and incremental partitions and loads the required historical data. Depending on the amount of data, this initial refresh can take considerable time.

Subsequent refreshes are generally much faster because only the partitions within the configured refresh window are refreshed.

For a 3 GB model, I would therefore test the initial refresh duration and memory/capacity requirements before moving it to production.

Another option: Fabric Lakehouse / Direct Lake

If you are already using Microsoft Fabric or are considering moving the data platform to Fabric, another architecture to evaluate is Lakehouse + Direct Lake.

With Direct Lake, the semantic model can consume Delta/Parquet data from OneLake without importing and duplicating the data into the semantic model. This can be particularly useful for large datasets and frequently changing source data.

So, broadly:

Current Import model → Incremental Refresh
Good option when you have a reliable date column and can define a safe refresh window for updates/deletions.

Large/frequently changing data + Fabric available → Lakehouse + Direct Lake
Worth considering if you are looking at a longer-term Fabric architecture.

References:

• Configure incremental refresh for Power BI semantic models
• Incremental refresh and real-time data overview
• Direct Lake overview
• Create a semantic model from a Fabric Lakehouse

Hope this helps.

Thanks!

Hi powerbiexpert22​ 

Adding to the above krishnakanth240​ suggestions :

I'd prioritize:

Phase 1

1. Find the largest fact tables contributing to the 3 GB model.
2. Confirm which tables actually need Incremental Refresh.
3. Determine the maximum historical correction/deletion period.
4. Apply Incremental Refresh only to appropriate large fact tables.
5. Keep smaller dimension/reference tables on normal refresh.

Phase 2, only if necessary

If historical changes have no time limit, introduce:

Source → SQL/Lakehouse → Power BI

and implement proper change/deletion handling there.

Phase 3, advanced

Use PPU + XMLA when you need targeted historical partition management rather than continually widening the normal refresh window. XMLA provides advanced partition-management capabilities.

Bottom line

Don't build CDC first. First determine the historical change/deletion window.

If the business says," Anything older than 12 months never changes," your problem becomes much easier. Incremental Refresh with a 12-month refresh window may be all you need.

If the business says," We can delete a five-year-old transaction tomorrow," then I would move to the staging + change tracking + controlled Power BI partition refresh architecture.

Hi powerbiexpert22​ 

Incremental refresh does not detect deletes. Rows deleted from source will be in model partition and Detect data changes only catches the deletes

You can add IsDeleted and LastModified columns at source. Filter out deleted rows in Power Query with IsDeleted = 'false' and enable Detect data changes on LastModified so partition refreshes when row is flagged as deleted. Increase refresh window to see how far back deletes can happen as older partitions are not reloaded

Once published with incremental refresh then .pbix can not be downloaded back from service so keep a development copy and use Tabular Editor for model changes

https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview

https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-configure