Forum Discussion
Incremental Refresh – Archived Partition Overwritten After Refresh
- 4 months ago
Hi JasonKang
Thank you for reaching out to the Microsoft Fabric Forum Community.
Juan-Power-bi Thanks for the inputs.
As per my knowledge, the issue isn’t publishing anymore, it’s how semantic_model_date is being used.Since you’re reassigning this date to different batches every day, the same partition ends up holding different data over time.
Incremental refresh expects that date column to stay consistent so it can manage partitions properly. In your case, because the value keeps changing for different data, Power BI ends up reprocessing that partition during refresh. When that happens, the partition gets rebuilt with the new data, which replaces what was already there instead of keeping the archived data.
Also, there’s no row count or data size limit in Fabric that would cause silent failures, if a partition returns 0 rows, it’s still treated as a successful refresh.
On top of that, since Parquet/Blob sources don’t support query folding, Power BI evaluates data based on the current files. If the source only contains the latest batch, partitions being refreshed may return no data and get overwritten.
So the behavior you’re seeing is actually expected. The refresh is working as designed, but the partition logic breaks because the date being used isn’t stable. Ideally, the column used for RangeStart and RangeEnd should be something fixed (like the actual record date), not something that changes per batch.
If there are any deviations from your expectation please let us know we are happy to address.
Thanks.
I think I see the issue. When you publish a new version of the .pbix from Desktop on Day 2, that overwrites the entire semantic model including all partitions, the archive you built on Day 1 gets wiped. Publishing from Desktop always replaces everything.
To preserve archive partitions across days, you need to only trigger refreshes from the Service after the initial publish, never republish from Desktop again. The incremental refresh engine in the Service knows to leave archive partitions alone and only touch the incremental window ,but only if the model already exists in the Service and you're refreshing it in place.
- JasonKang4 months agoRegular Visitor
Hi Juan,
Thank you for your response. I understand that publishing from Desktop will overwrite the existing semantic model. Therefore, I updated the semantic model using the update functionality in the Power BI workspace (as shown in the screenshot).
I have also attached a screenshot showing the workspace type for your reference.