Forum Discussion
Responsive ResidentImplementing Medallion Architecture in Fabric : Bronze Layer ingesting data from a SQL Server
Dear colleagues,
After having read dozens of documents detailing what are the principles of the Medallion Arhitecture I am a bit confused about how to make it "effective" on Fabric. I have many doubts with regards the Bronze and the famous landing zone.
Imagine an scenario where the source data is a simple RDBMS placed on a on premise SQL server, We need to import some tables. How should we apply the "guidelines" of the medallion architecture in such a case?
Should use a Lakehouse for managing the landing zone (files folder) and the ingestion zone (tables folder)? If yes, should I store a copy of each of SQL sever tables on the files folder (one folder by entity, a file by name e.g: Table1_DDMMYYYY) ?
Regards,
Alfons
Hi alfBI if you take a look at this document there is a section "Medallion architecture in Fabric" which says "In a typical medallion architecture implementation in Fabric, the bronze zone stores the data in the same format as the data source. When the data source is a relational database, Delta tables are a good choice"
https://learn.microsoft.com/en-us/fabric/onelake/onelake-medallion-lakehouse-architecture
Are you looking at creating a pure landing zone before your bronze/raw zone? Depending on your source data it may not be necessary to have a landing zone.
2 Replies
- AndyDDC
Most Valuable Professional
Hi alfBI if you take a look at this document there is a section "Medallion architecture in Fabric" which says "In a typical medallion architecture implementation in Fabric, the bronze zone stores the data in the same format as the data source. When the data source is a relational database, Delta tables are a good choice"
https://learn.microsoft.com/en-us/fabric/onelake/onelake-medallion-lakehouse-architecture
Are you looking at creating a pure landing zone before your bronze/raw zone? Depending on your source data it may not be necessary to have a landing zone.
daircomResolver II
AndyDDC ,
If I move a file to the bronze layer I should transform it into a table right? (using a notebook for example)
Is the table also supposed to be in the bronze layer (so that both the file and the table it is based on are in the bronze layer), or should the file be in the bronze layer and the table based on the file in the silver layer?