Forum Discussion
Post ProdigyDeep dive on open mirroring
Hi pmscorca ,
Thank you for the clarification.
From a practical perspective, Open Mirroring in Microsoft Fabric works more like a managed CDC/replication service than a traditional ETL pipeline. The source system writes change data into a landing zone, and the Fabric mirroring engine continuously synchronizes and maintains Delta tables inside One Lake.
Compared to pipelines, Open Mirroring automatically handles incremental synchronization, change tracking, schema updates, and continuous replication, reducing the need for scheduled ETL jobs and manual orchestration.
Another key advantage is that the mirrored data becomes immediately available for Power BI, SQL analytics, Spark/Data Engineering, and AI/Data Science workloads because the data is maintained in Delta Parquet format.
In general, Open Mirroring is better suited for near real-time replication scenarios, while pipelines are better for complex ETL transformations and orchestration.
I hope this helps. Please feel free to reach out if you have any further questions.
Community SupportHi pmscorca ,
Thank you for reaching out to the Microsoft Community Forum.
Open Mirroring in Microsoft Fabric is used to continuously replicate data from external systems into OneLake without building complex ETL pipelines. Fabric automatically detects data changes (CDC/incremental updates), handles inserts/updates/deletes, and converts the data into Delta Parquet format for analytics.
Compared to traditional ingestion methods, Open Mirroring reduces the need to manage scheduled ETL jobs, incremental logic, schema changes, and manual orchestration. Instead of manually building and maintaining pipeline-based data movement, Fabric continuously syncs the latest source changes automatically into OneLake.
This makes the data quickly available for Power BI, SQL Analytics, Spark/Data Engineering, and AI workloads.
Official documentation:
https://learn.microsoft.com/en-us/fabric/mirroring/open-mirroring
I hope this helps. Please feel free to reach out if you have any further questions.
Post ProdigyHi, thanks but I hope in a more detailed reply compared to what is reported in the official documentation, perhaps derived from experiences.
Thanks
- v-anbandari
Hi pmscorca ,
Thank you for the clarification.
From a practical perspective, Open Mirroring in Microsoft Fabric works more like a managed CDC/replication service than a traditional ETL pipeline. The source system writes change data into a landing zone, and the Fabric mirroring engine continuously synchronizes and maintains Delta tables inside One Lake.
Compared to pipelines, Open Mirroring automatically handles incremental synchronization, change tracking, schema updates, and continuous replication, reducing the need for scheduled ETL jobs and manual orchestration.
Another key advantage is that the mirrored data becomes immediately available for Power BI, SQL analytics, Spark/Data Engineering, and AI/Data Science workloads because the data is maintained in Delta Parquet format.
In general, Open Mirroring is better suited for near real-time replication scenarios, while pipelines are better for complex ETL transformations and orchestration.
I hope this helps. Please feel free to reach out if you have any further questions.
- pmscorca
Hi, but how is data writing implemented in landing zone?
Thanks
- v-anbandari
Hi pmscorca , Thanks for the follow-up.
In Open Mirroring, data writing to the landing zone is handled externally using a file-based approach.
- The source system (or connector/custom app) writes change data files directly into the One Lake landing zone using the required folder structure.
- These files are typically in Parquet/CSV format and include insert, update, and delete markers, along with a _metadata.json file to define key columns.
Once the data is written:
- The Fabric mirroring engine continuously monitors the landing zone
- It reads incoming files, interprets the change (CDC) information, and
- Applies those changes to Delta tables automatically in One Lake
For your reference:
Open Mirroring Landing Zone Requirements and Formats - Microsoft Fabric | Microsoft LearnI hope this clarifies .Please feel free to reach out to us if any queries.
Thank you.