Patterns: Medallion Architecture within Azure Fabric Hub-and-Spoke and workspace/domain design

[TwoCents]

Hello Team / Community,

If a large organization is migrating towards Azure Fabric and establishing a data mesh / domain-oriented architecture structured as a hub-and-spoke model. We currently possess a significant amount of data in various existing Azure Storage Accounts (ADLS Gen2), often organized by source system or business domain.

A core requirement is enabling domain owners to integrate relevant data from their existing source ADLS into their respective Fabric domains. We intend to use Fabric shortcuts to link directly to specific paths within the source ADLS, facilitating a controlled and managed way to access this external data.

Furthermore, within each domain, we may plan to adopt the Medallion Architecture concept (Bronze, Silver, Gold layers) to structure data processing and refinement.

We are now designing how to map these concepts – the external ADLS sources, the Fabric Domains, Workspaces, and Lakehouses – together to realize this architecture. We are seeking architectural patterns and practical guidance on this mapping and the resulting data flow.

Specifically, we'd deeply appreciate insights on:

1. Connecting Source ADLS to Bronze (via Shortcuts): What are the recommended patterns and considerations for using Fabric shortcuts to link to specific sub-folders or paths within our existing source ADLS Gen2 containers? How does this external link via shortcut represent or feed the Bronze layer within a domain's Medallion structure?
2. Mapping Medallion to Fabric Components: How do the Bronze, Silver, and Gold layers of the Medallion Architecture typically map to Fabric Workspaces and Lakehouses within a single domain in this hub-and-spoke model?
   • Are the layers separated into different Lakehouses (e.g., one Lakehouse for Bronze, one for Silver, one for Gold per domain)?
   • Or is the Medallion structure typically implemented within a single Lakehouse using different folder structures or schemas (e.g., /Files/Bronze, /Tables/Silver, /Tables/Gold)?
   • What are common patterns for utilizing multiple Workspaces within a domain to support the Medallion flow (e.g., separate workspaces for ingestion, transformation, consumption)? How many workspaces per domain are typically seen for this?
3. End-to-End Data Flow & Promotion: What are the recommended patterns for the data movement and controlled promotion of data:
   • From the external source (accessed via shortcut in the Bronze concept)
   • Into the Silver layer (cleansed, integrated)
   • Then into the Gold layer (aggregated, consumption-ready)
   • All managed within the domain's defined Workspaces/Lakehouses? What Fabric items (Data Pipelines, Notebooks, etc.) are best suited for managing this flow between layers?

We are looking for practical, pattern-based guidance on how to align our existing ADLS sources, the Fabric hub-and-spoke domain structure, and the internal Medallion layering using Workspaces and Lakehouses effectively and scalably.

Any shared experiences, architectural diagrams, or specific patterns implemented in similar scenarios would be extremely valuable.

Thank you for your time and expertise!

[v-sathmakuri]

Hi @TwoCents ,

I hope this information provided is helpful. Feel free to reach out if you have any further questions or would like to discuss this in more detail. If responses provided answers your question, please accept it as a solution so other community members with similar problems can find a solution faster.

Thank you!!

[v-sathmakuri]

Hi @TwoCents ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If the responses has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you!!

[v-sathmakuri]

Hi @TwoCents ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[v-sathmakuri · Accepted solution]

Hi @TwoCents ,

Thank you for reaching out to Microsoft Fabric Community.

When you are building a Fabric hub and spoke model with domains (like Finance, Sales, etc.), and you have a lot of existing data sitting in Azure Data Lake (ADLS Gen2), the best way to bring that data into Fabric is by using shortcuts.
Each domain should create shortcuts inside their own Lakehouse pointing directly to just the folders they need from ADLS. This becomes the Bronze layer (raw data).

For organizing your Medallion Architecture (Bronze, Silver, Gold), the simplest and cleanest way is to use one Lakehouse per domain and split it into folders like /Files/Bronze, /Tables/Silver, /Tables/Gold.
Only if your data grows massive (like many terabytes) you can consider separating Lakehouses for each layer, but that's rare early on.

Each domain usually needs just one workspace at the start. You don't need to split workspaces unless you grow really big.
Inside the domain workspace, you handle:

• Shortcuts for Bronze (raw external data),
• Pipelines/Notebooks to cleanse into Silver,
• More Pipelines/Notebooks to refine into Gold.

Data movement happens using Fabric Pipelines and Spark Notebooks. Pipelines help automate the steps and Notebooks are great if you have complex logic (like heavy data transformations or machine learning).

If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it! 

Thank you!!