Forum Discussion
vijith_mar
1 day agoFrequent Visitor
Validating a metadata-driven Bronze → Silver incremental load pattern in Fabric
Hi all, I’m building a metadata-driven Bronze → Silver ingestion framework in Microsoft Fabric and would like feedback on the approach before scaling it to a few hundred tables. Setup Source: Dyn...
v-achippa
Community Support
1 day agoHi vijith_mar,
Thank you for reaching out to Microsoft Fabric Community.
Your overall approach looks reasonable. Use a watermark per table, deduplicate new Bronze records and merge them into Silver. Update the watermark only after the Delta MERGE succeeds. Use CDC or a source modified-date column to capture updates and make sure source deletions are provided through CDC or a soft-delete indicator. Test incremental loads, retries, late-arriving records and deletions before scaling the pipeline to 200+ tables.
For reference, follow the below documentation:
Incrementally load data from Data Warehouse to Lakehouse - Microsoft Fabric | Microsoft Learn
Thanks and regards,
Anjan Kumar Chippa