Forum Discussion
Deep dive on open mirroring
Hi,
I would like to better understand which mechanisms, services and functions are behind the operation of open mirroring to better understand the added value compared to an ingestion via f.e. pipeline.
Any helps to me, please? Thanks
Hi pmscorca ,
Thank you for the clarification.
From a practical perspective, Open Mirroring in Microsoft Fabric works more like a managed CDC/replication service than a traditional ETL pipeline. The source system writes change data into a landing zone, and the Fabric mirroring engine continuously synchronizes and maintains Delta tables inside One Lake.
Compared to pipelines, Open Mirroring automatically handles incremental synchronization, change tracking, schema updates, and continuous replication, reducing the need for scheduled ETL jobs and manual orchestration.
Another key advantage is that the mirrored data becomes immediately available for Power BI, SQL analytics, Spark/Data Engineering, and AI/Data Science workloads because the data is maintained in Delta Parquet format.
In general, Open Mirroring is better suited for near real-time replication scenarios, while pipelines are better for complex ETL transformations and orchestration.
I hope this helps. Please feel free to reach out if you have any further questions.
7 Replies
Hi pmscorca ,
Thank you for reaching out to the Microsoft Community Forum.
Open Mirroring in Microsoft Fabric is used to continuously replicate data from external systems into OneLake without building complex ETL pipelines. Fabric automatically detects data changes (CDC/incremental updates), handles inserts/updates/deletes, and converts the data into Delta Parquet format for analytics.
Compared to traditional ingestion methods, Open Mirroring reduces the need to manage scheduled ETL jobs, incremental logic, schema changes, and manual orchestration. Instead of manually building and maintaining pipeline-based data movement, Fabric continuously syncs the latest source changes automatically into OneLake.
This makes the data quickly available for Power BI, SQL Analytics, Spark/Data Engineering, and AI workloads.
Official documentation:
https://learn.microsoft.com/en-us/fabric/mirroring/open-mirroringI hope this helps. Please feel free to reach out if you have any further questions.
Hi, thanks but I hope in a more detailed reply compared to what is reported in the official documentation, perhaps derived from experiences.
Thanks
Hi pmscorca ,
Thank you for the clarification.
From a practical perspective, Open Mirroring in Microsoft Fabric works more like a managed CDC/replication service than a traditional ETL pipeline. The source system writes change data into a landing zone, and the Fabric mirroring engine continuously synchronizes and maintains Delta tables inside One Lake.
Compared to pipelines, Open Mirroring automatically handles incremental synchronization, change tracking, schema updates, and continuous replication, reducing the need for scheduled ETL jobs and manual orchestration.
Another key advantage is that the mirrored data becomes immediately available for Power BI, SQL analytics, Spark/Data Engineering, and AI/Data Science workloads because the data is maintained in Delta Parquet format.
In general, Open Mirroring is better suited for near real-time replication scenarios, while pipelines are better for complex ETL transformations and orchestration.
I hope this helps. Please feel free to reach out if you have any further questions.
