Forum Discussion

Lanceometer's avatar
Lanceometer
Icon for Helper I rankHelper I
4 months ago
Solved

Mirrored Snowflake Database Storage

Hi Folks,   Some info upfront: - We use a Fabric F16 Capacity - I know the F16 Capacity offers 16TB free mirroring Storage   I was trying the Mirrored Snowflake Database. It worked well. It was...
capacities
governance
one lake
  • Dev_Dholakia's avatar
    Dev_Dholakia
    4 months ago

    Hey Lanceometer ,

     

    You have diagnosed it right, this is a source-side pattern, not a Fabric bug.

    • Why storage grows: Fabric mirroring uses Snowflake Streams (CDC). A nightly full rebuild makes every row look new, so a fresh set of Parquet files gets written each cycle while older Delta versions stick around
    • 1 day retention: It's only a threshold for when old files become eligible for cleanup, not  guarantee that vacuum runs daily
    • Manual VACUUM: Not supported on a mirrored database. Maintenance is handled internally (OPTIMIZE runs roughly every ~50 writes per table)

    Next steps:

    • Real fix: ask the Snowflake team to switch to incremental MERGE/UPDATE instead of a nightly rebuild
    • Otherwise, skip mirroring for this table — use a OneLake shortcut or a pipeline load
    • Keep retention at 1 day and give it a couple of cycles to plateau

    Useful links:

    Mirroring Snowflake in Fabric
    VACUUM Delta tables

    Found this useful? A quick Kudos goes a long way.

    Got what you needed? Marking this as the Accepted Solution helps others land on the right answer faster when they search for the same thing

Omkar_1712's avatar
Omkar_1712
Icon for Solution Sage rankSolution Sage
4 months ago

Hello Lanceometer,

From your investigation, your assumption sounds reasonable.


If the source process rebuilds the entire Snowflake table each night (rather than performing incremental updates), Fabric Mirroring may interpret this as a full set of changes, resulting in new Delta files being created and increased storage consumption.


Regarding the Delta Table Management settings:

  • The retention threshold controls when obsolete Delta files become eligible for cleanup, but it doesn't necessarily mean they are removed immediately after the threshold is reached.

  • Cleanup (VACUUM) is managed by the Fabric service, and as far as I'm aware, there isn't currently a supported way to manually trigger a VACUUM operation or force cleanup for mirrored databases.

If the storage continues to grow well beyond the expected retention period, despite the retention threshold being set to one day, it may indicate that:

  • Background cleanup hasn't completed yet, or

  • The continuous full-table rebuilds are generating Delta files faster than the retention process can reclaim them.

It would also be worth confirming whether the Snowflake ingestion pattern can be changed to perform incremental updates instead of full table rebuilds, as this would significantly reduce both storage growth and processing overhead.


If the storage keeps increasing without stabilizing after several retention cycles, I'd recommend opening a Microsoft support ticket. This would help determine whether the observed growth is expected behavior for mirrored Snowflake databases or if there's an issue with the Delta cleanup process.


Best regards,
Omkar Shinde
Microsoft Fabric Enthusiast | Power BI Consultant

💡 If you found this response helpful, please consider giving it a Kudos.
✅ If this resolves your question, please mark it as the Accepted Solution to help others in the community.