Open Mirroring in Microsoft Fabric: A Step-by-Step Hands-On Guide

[suparnababu8]

In this blog, I’ll walk you through Open Mirroring in Microsoft Fabric using a practical, hands-on example, covering initial load, incremental updates, and schema changes.
Let’s Dive in

What is Open Mirroring in Microsoft fabric?

Open Mirroring is a new Fabric capability that allows any application to write change data directly into a Mirrored Database using open formats like Delta Lake.

• Enables ingestion of application data into OneLake via mirroring
• Supports data landing through APIs or manual file uploads (UI-based)
• Simplifies handling of inserts, updates, and schema changes
• Keeps mirrored data continuously updated and analytics-ready

Unlike traditional ingestion pipelines, Open Mirroring removes the need to manage connectors or orchestration workloads.

Pre-requisites:

Before getting started, ensure you have:

• A Microsoft Fabric Paid or Free Trial account
• Basic SQL knowledge

For this demo, I used an Employee dataset. Source Files (Parquet files) containing employee data and updates.

Step 1: Create a Mirrored Database

I’ve created workspace ‘Open-Mirroring-WS’ and click on ‘New item’ and select ‘Mirrored database’

Provide name (Emp_MirroredDataBase) and click on Create

It’s creating Mirrored Database

Fabric provisions the mirrored database automatically.

Step 2: Initial Data Load

Open the mirrored database and click on Upload Files

I will upload ‘EmployeeData.parquet’ from my local system and I will do this by clicking browse files

Now select   ‘EmployeeData.parquet’ from my local machine

Now I will provide details such as Table name(empdata), Primary Key Column and you will be able to view the preview data. Then click ‘Create Table’

Once completed, Fabric initializes the table and prepares it for mirroring.

Step 3: Refresh and Validate Data

Click on ‘Refresh’ , with in few minutes You will notice data is landed under ‘empdata’ folder and Status updated with Rows Replicated and Last Completed Time.

To validate, click on Mirrored database dropdown and select SQL analytics endpoint

Query the table and confirm the data is available

Step 4: Handling Data Updates Using __rowMarker__

When records change, Open Mirroring uses a special column called __rowMarker__ to identify the type of operation.

__rowMarker__ Overview:

• Helps Fabric understand row-level changes
• Ensures updates are applied correctly during incremental loads

After updating employee salaries, I uploaded EmployeeData_Update1.parquet containing modified rows with __rowMarker__ populated.

Now uploading ‘EmployeeData_Update1.parquet’ file having updated records

Now select Update existing table and choose ‘empdata’, you will be seeing the data preview and click on update

Click on Refresh and within few minutes you will notice that change in Rows replicated and Last Completed Time.

Querying the table confirms the salary updates were applied successfully

Step 5: Updating Existing Column Values

Next I updated Marital status and Gender columns (expanded full names)

Now I’m uploading ‘EmployeeData_Update2.parquet’ file having updated records

Now select Update existing table and choose ‘empdata’, you will be seeing the data preview and click on update

Click on Refresh and after couple of minutes you will see the change in Rows replicated and Last Completed Time.

Querying the table confirms the Marital status and Gender columns updates were applied successfully

Step 6: Handling Schema Changes (Column Addition)

To test schema evolution, I added a new column called ‘ExitDate’

Now I’m uploading ‘EmployeeData_ColumnAdd.parquet’ file having new column addition

Now select Update existing table and choose ‘empdata’, you will be seeing the data preview and click on update

Click on Refresh and after couple of minutes you will see the change in Rows replicated and Last Completed Time.

Querying the table confirms the new column ‘ExitDate’ columns updates were applied successfully and rows without exit dates were automatically populated with NULL

This demonstrates Open Mirroring’s ability to handle schema changes with minimal effort.

Once data is mirrored:

• Create a Semantic Model on top of the table
• Use it for Power BI reporting
• Any future data changes are reflected after refresh
• The process can be fully automated for business reporting

Advantages of Open Mirroring

• Free storage for mirrored data based on Fabric capacity
• Example: An F32 capacity provides 32 TB of free mirroring storage
• OneLake storage is billed only when free limits are exceeded
• Near real-time data replication
• No connection setup or pipeline management required

Current Limitations

• Supports only Parquet and delimited text files (CSV)
• Renaming a table requires dropping and recreating it
• Changing a column data type requires reloading initial and incremental data

Conclusion:

Open Mirroring in Microsoft Fabric is a simple yet powerful way to ingest and keep data up to date in OneLake. It eliminates complex ingestion pipelines, supports schema evolution, and integrates seamlessly with analytics and reporting layers.

For organizations looking for near real-time data availability with minimal operational overhead, Open Mirroring is an excellent choice.

For a visual explanation, please refer to the detailed video : https://youtu.be/CtoDdjpdjT4?si=2JneN1hxs6wCcD1b

— Inturi Suparna Babu

LinkedIn 

[rajendraongole1]

Fantastic Babu!

Good work 

[krishnakanth240]

Thanks for sharing Open Mirroring blog @suparnababu8!

[NagaSuresh]

Thanks for Sharing @suparnababu8 

[Mvsainathareddy]

Thanks for sharing 😊 great job

@suparnababu8 

[Ugk161610]

Thanks for sharing @suparnababu8