Forum Discussion

JasonKang's avatar
JasonKang
Regular Visitor
4 months ago
Solved

Incremental Refresh – Archived Partition Overwritten After Refresh

ENVIRONMENT Power BI with Microsoft Fabric capacity Data source: Parquet files stored in Azure Blob Storage (~100MB) Incremental refresh setting: 2 years archive, 8 days incremental INCREMENTAL...
  • v-priyankata's avatar
    4 months ago

    Hi JasonKang 

    Thank you for reaching out to the Microsoft Fabric Forum Community.

    Juan-Power-bi Thanks for the inputs.


    As per my knowledge, the issue isn’t publishing anymore, it’s how semantic_model_date is being used.

    Since you’re reassigning this date to different batches every day, the same partition ends up holding different data over time.

    Incremental refresh expects that date column to stay consistent so it can manage partitions properly. In your case, because the value keeps changing for different data, Power BI ends up reprocessing that partition during refresh. When that happens, the partition gets rebuilt with the new data, which replaces what was already there instead of keeping the archived data.

    Also, there’s no row count or data size limit in Fabric that would cause silent failures, if a partition returns 0 rows, it’s still treated as a successful refresh.

    On top of that, since Parquet/Blob sources don’t support query folding, Power BI evaluates data based on the current files. If the source only contains the latest batch, partitions being refreshed may return no data and get overwritten.

    So the behavior you’re seeing is actually expected. The refresh is working as designed, but the partition logic breaks because the date being used isn’t stable. Ideally, the column used for RangeStart  and  RangeEnd should be something fixed (like the actual record date), not something that changes per batch.

     

    If there are any deviations from your expectation please let us know we are happy to address.

    Thanks.