Forum Discussion
Using the Mirroring feature in a data warehouse project
Hi,
I need to design a data warehouse to feed with data coming from some database sources.
Could be a good thing using the Mirroring feature at bronze layer instead of implementing a lakehouse in order to accelerate the raw ingestion?
Thanks
Yes, Mirroring is a great accelerator for bronze if you want that layer to be raw, untransformed data from your source environment. I've used mirroring in a medallion architecture with good success. At that point it's Delta tables sitting in your OneLake, which can be consumed for processes to move your data into silver.
7 Replies
When you refer to lakehouse, I am assuming you mean lakehouse shortcuts. If I am incorrect on that assumption, please clarify.
Since the data would be replicated and delta tables created in OneLake, there's a chance you will get better performance from mirroring when ingesting your data into your silver layer.
For each data source, be sure to check that the source is supported by current shortcut/mirroring options. In most situations, your source would either be supported by shortcuts or Mirroring, but not both. Also be sure to check that your source is supported by mirroring, as current mirroring options focus on specific sources and have limitations around networking.
If the question is just 'Is Mirroring good for a bronze layer instead of writing processes to put it in a lakehouse?', I'd say the answer is yes. You'll save significant dev time mirroring your data than building Copy Jobs or similar processes to do the same. Just be sure to check performance impact on your source environments, as Mirroring is essentially performing CDC operations on your source data.
If this post helps, please consider accept as solution to help other members find it more quickly.
Hi,
when I talk about a lakehouse I refer to the object and not to the related shortcut.
I'd like to understand if Mirroring feature could be really useful in a medallion architecture to implement a data warehouse: it seems to be a valid alternative respect to a lakehouse in the bronze layer; so, Mirroring feature could be an accelerator component to deliver a data warehouse solution, isn't it?
Next, at silver layer I could have a lakehouse that reads from the bronze mirrored data source and so on.
I've already read the documentation about this feature.
Thanks
Yes, Mirroring is a great accelerator for bronze if you want that layer to be raw, untransformed data from your source environment. I've used mirroring in a medallion architecture with good success. At that point it's Delta tables sitting in your OneLake, which can be consumed for processes to move your data into silver.
HI pmscorca,
I'd like to suggest take a look at the document of mirroring feature at first:
Mirroring - Microsoft Fabric | Microsoft Learn
Regards,
Xiaoxin Sheng
