Incremental Refresh – Archived Partition Overwritten After Refresh

[JasonKang]

ENVIRONMENT

Power BI with Microsoft Fabric capacity
Data source: Parquet files stored in Azure Blob Storage (~100MB)
Incremental refresh setting: 2 years archive, 8 days incremental

INCREMENTAL REFRESH RANGES (using 2026/03/30 as the reference date)

Archive range: 2024/01/01 to 2026/03/29 (not refreshed)
Incremental range: 2026/03/30 to 2026/04/06 (refreshed every time)

DESIGN OVERVIEW

I have two date columns:

1. "date" - the actual date of each record, used to control which batch of historical data is exported from BigQuery

2. "semantic_model_date" - a fixed value assigned to each batch, used to map to Power BI's RangeStart / RangeEnd and control which day within the incremental range this batch falls into

Because the historical data volume is too large to import all at once, I use the following rolling strategy to gradually load 2 years of historical data into the archive:

Day 1 (executed on 4/6):
- Incremental range = 3/30 to 4/6
- Export records where date = 1/1 to 1/5
- semantic_model_date = 3/30 (earliest day of the incremental range)
- Publish from Desktop to Fabric workspace

Day 2 (executed on 4/7):
- Incremental range slides to 3/31 to 4/7
- 3/30 partition moves into the archive, so 1/1 to 1/5 data should now be archived
- Export records where date = 1/6 to 1/10
- semantic_model_date = 3/31 (new earliest day of the incremental range)
- Trigger refresh from Fabric workspace

Each day the window slides by one day, archiving one batch at a time, gradually loading all 2 years of historical data into the archive.

POWER QUERY SETUP

- "Filtered Rows" is the second step, immediately after "Source"
- Filter condition: [semantic_model_date] >= RangeStart and [semantic_model_date] < RangeEnd
- No errors or warnings appear during or after refresh in the Service

PROBLEM

After triggering a refresh from the workspace, the 2026/03/29 partition disappears and only 2026/03/30 data remains. The archived partition is overwritten instead of being preserved.

EXPECTED BEHAVIOR

- 2026/03/29 should be in the archive partition (2024/01/01 to 2026/03/29) and remain untouched after refresh
- 2026/03/30 should be added to the incremental partition (2026/03/30 to 2026/04/06)
- Both dates should coexist

WHAT'S STRANGE

- An identical setup with only 2 columns on a smaller dataset works correctly. Both 03/29 and 03/30 are preserved after refresh.
- Even when the 10+ column table is reduced to the same 2 columns as the working table, the issue still occurs.
- This suggests the problem is related to data content or row count, not column count.
- The refresh history shows success with no errors or warnings.

QUESTIONS

1. Why does the archived partition (2024/01/01 to 2026/03/29) get silently overwritten even though refresh completes successfully?
2. Is there a row count or data size threshold in Fabric that causes partition writes to fail silently?
3. Are there any known issues with incremental refresh on Parquet files stored in Azure Blob Storage?
4. Is there a fundamental problem with this rolling batch strategy for loading historical data into the archive?

[v-priyankata · Accepted solution]

Hi @JasonKang 

Thank you for reaching out to the Microsoft Fabric Forum Community.

@Juan-Power-bi Thanks for the inputs.

As per my knowledge, the issue isn’t publishing anymore, it’s how semantic_model_date is being used.

Since you’re reassigning this date to different batches every day, the same partition ends up holding different data over time.

Incremental refresh expects that date column to stay consistent so it can manage partitions properly. In your case, because the value keeps changing for different data, Power BI ends up reprocessing that partition during refresh. When that happens, the partition gets rebuilt with the new data, which replaces what was already there instead of keeping the archived data.

Also, there’s no row count or data size limit in Fabric that would cause silent failures, if a partition returns 0 rows, it’s still treated as a successful refresh.

On top of that, since Parquet/Blob sources don’t support query folding, Power BI evaluates data based on the current files. If the source only contains the latest batch, partitions being refreshed may return no data and get overwritten.

So the behavior you’re seeing is actually expected. The refresh is working as designed, but the partition logic breaks because the date being used isn’t stable. Ideally, the column used for RangeStart  and  RangeEnd should be something fixed (like the actual record date), not something that changes per batch.

If there are any deviations from your expectation please let us know we are happy to address.

Thanks.

[v-priyankata]

Hi @JasonKang 

Thank you for reaching out to the Microsoft Fabric Forum Community.

I hope the information provided was helpful. If you still have questions, please don't hesitate to reach out to the community.

[v-priyankata]

Hi @JasonKang 

Hope everything’s going smoothly on your end. I wanted to check if the issue got sorted. if you have any other issues please reach community.

[Juan-Power-bi]

I think I see the issue. When you publish a new version of the .pbix from Desktop on Day 2, that overwrites the entire semantic model including all partitions,  the archive you built on Day 1 gets wiped. Publishing from Desktop always replaces everything.
To preserve archive partitions across days, you need to only trigger refreshes from the Service after the initial publish, never republish from Desktop again. The incremental refresh engine in the Service knows to leave archive partitions alone and only touch the incremental window ,but only if the model already exists in the Service and you're refreshing it in place.

[JasonKang]

Hi Juan,

Thank you for your response. I understand that publishing from Desktop will overwrite the existing semantic model. Therefore, I updated the semantic model using the update functionality in the Power BI workspace (as shown in the screenshot).

I have also attached a screenshot showing the workspace type for your reference.