Forum Discussion

jashfabric's avatar
jashfabric
Helper II
2 years ago
Solved

Implementing Medallion Architecture in Microsoft Fabric

Hi,
I'm working on implementing a Medallion Architecture for data warehousing in Microsoft Fabric. I'm particularly interested in understanding best practices for acheiving Medallion Architecture between the Bronze, Silver, and Gold layers.
I would like to know what is the best process to setup a medallion architecture?

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi jashfabric 
    Thanks for using Fabric Community.
    The Medallion Architecture is a powerful design pattern used to logically organize data in a lakehouse. It helps you build a single source of truth for enterprise data products by organizing data into distinct layers or zones.
    Here’s how you can set up Medallion Architecture in Microsoft Fabric:

     

    a. Create a Lakehouse:

    • Start by provisioning a lakehouse in Fabric. The lakehouse is the foundation for Medallion Architecture.
    • Fabric provides OneLake, a unified, logical data lake for your entire organization. It’s automatically provisioned with every Fabric tenant.
    • OneLake serves as the single location for all your analytics data, reducing silos and data movement.

    b. Organize Data into Zones:

    • Within OneLake, create three top-level folders (zones) corresponding to the Medallion layers: Bronze, Silver, and Gold.
    • Each zone represents a different level of data quality.

    c. Data Movement and Transformation:

    • Ingest raw data into the Bronze zone. This data can come from various sources.
    • Use Fabric’s capabilities (such as notebooks, pipelines, and dataflows) to validate, cleanse, and transform data.
    • Move validated data to the Silver zone.

    d. Enrichment and Optimization:

    • In the Silver zone, perform additional transformations and calculations.
    • Enrich data with context, business rules, and metadata.
    • Move enriched data to the Gold zone.

    You can adapt the specific tools and processes based on your unique data landscape and requirements. Please refer to these links for more information:

    https://learn.microsoft.com/en-us/fabric/onelake/onelake-medallion-lakehouse-architecture
    https://medium.com/@mariusz_kujawski/exploring-the-medallion-architecture-in-microsoft-fabric-f19ad7590293
    https://blog.det.life/what-is-medallion-architecture-and-how-can-you-use-it-in-fabric-6b0056ababc4

    Hope this helps. Please let me know if you have any further questions.

     

2 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi jashfabric 
    Thanks for using Fabric Community.
    The Medallion Architecture is a powerful design pattern used to logically organize data in a lakehouse. It helps you build a single source of truth for enterprise data products by organizing data into distinct layers or zones.
    Here’s how you can set up Medallion Architecture in Microsoft Fabric:

     

    a. Create a Lakehouse:

    • Start by provisioning a lakehouse in Fabric. The lakehouse is the foundation for Medallion Architecture.
    • Fabric provides OneLake, a unified, logical data lake for your entire organization. It’s automatically provisioned with every Fabric tenant.
    • OneLake serves as the single location for all your analytics data, reducing silos and data movement.

    b. Organize Data into Zones:

    • Within OneLake, create three top-level folders (zones) corresponding to the Medallion layers: Bronze, Silver, and Gold.
    • Each zone represents a different level of data quality.

    c. Data Movement and Transformation:

    • Ingest raw data into the Bronze zone. This data can come from various sources.
    • Use Fabric’s capabilities (such as notebooks, pipelines, and dataflows) to validate, cleanse, and transform data.
    • Move validated data to the Silver zone.

    d. Enrichment and Optimization:

    • In the Silver zone, perform additional transformations and calculations.
    • Enrich data with context, business rules, and metadata.
    • Move enriched data to the Gold zone.

    You can adapt the specific tools and processes based on your unique data landscape and requirements. Please refer to these links for more information:

    https://learn.microsoft.com/en-us/fabric/onelake/onelake-medallion-lakehouse-architecture
    https://medium.com/@mariusz_kujawski/exploring-the-medallion-architecture-in-microsoft-fabric-f19ad7590293
    https://blog.det.life/what-is-medallion-architecture-and-how-can-you-use-it-in-fabric-6b0056ababc4

    Hope this helps. Please let me know if you have any further questions.