topic Workspace & Medallion Architecture Design with Centralised Logic on Bronze and Silver in Data Engineering

Workspace & Medallion Architecture Design with Centralised Logic on Bronze and Silver

ndamoi — Mon, 04 Aug 2025 15:40:19 GMT

Hello all, I hope you're doing well!
This is my first post here, so please be gentle 😊 I'm a junior data engineer/scientist working at a small-to-midsized company, and I'm currently leading the creation of our data platform entirely on my own.

As you’ll see from the diagram, I’ve split responsibilities between Microsoft Fabric (for data engineering) and Power BI (for reporting and semantic modeling). I’d greatly appreciate feedback from experienced professionals on whether my architecture reflects common best practices, or if I may be steering toward complications later on.

Data platform architecture

Here the key points:

Centralized Bronze & Silver Workspaces:

I ingest raw data in a Bronze workspace (with separate lakehouses per source), using dedicated ingestion pipelines for each source.
The Silver layer lives in a centralized workspace where I apply shared validation, cleaning, and transformations that apply company-wide.
A significant portion of Silver tables (even from different sources) are reused across departments, which is why I opted to centralize these layers

Registry Workspace:
- Tracks ingestion and transformation pipeline runs.
- Monitors source-specific metadata (schemas, freshness, etc.).

Departmental Workspaces:
- Each department has its own workspace with:
- A Gold lakehouse for curated, domain-specific outputs.
- An intermediate Silver lakehouse that uses shortcuts to central Silver (for additional filters or enrichments relevant only to the department).

Environment Strategy:
- Every workspace is split into Dev / Test / Prod versions.
- Repos are mapped as follows (one repo per group dev,test,prod for each workspace):
- Dev → 'develop' branch
- Test → 'release x.xx' branch
- Prod → 'main' branch (release branch merged to once validated)
- For unsupported items, i imagine using fabric deployment pipelines as well

Power BI Reporting:
- Semantic models and reports are developed from Gold lakehouses in department-specific Power BI workspaces

My Questions

I’d love your insights on a few points I’m still unsure about:

Is centralizing Bronze and Silver layers a common and scalable practice for a small/midsized company, especially when tables are heavily shared?
Does the overall workspace architecture align with good practice — or am I missing anything critical from a governance or deployment point of view?
Is using intermediate Silver layers (via shortcuts) in department workspaces a smart compromise, or will it introduce unnecessary complexity for lineage, access, and metadata control?
Are there blind spots I might be overlooking (e.g., in security, performance tuning, or workflow design)?
Are the workplaces well structured for the specific usecase ? Keep in mind that i am the only one that will have access to bronze and silver workspaces, the departments have access to the power BI service workspaces and the gold lakehouse only.

Thanks in advance for any feedback — I really want to make sure I'm building something solid and maintainable before our data needs scale!

Re: Workspace & Medallion Architecture Design with Centralised Logic on Bronze and Silver

Anonymous — Tue, 05 Aug 2025 09:32:05 GMT

Hi @ndamoi ,

Thank you for reaching out to the Microsoft fabric community forum.

To answer your first question: yes, keeping the bronze and silver layers centralized is a common and smart setup, especially for small or mid-sized companies. It helps you keep your data consistent, makes things easier to manage, and avoids repeating the same transformation logic in different places which is great when different teams use the same cleaned data.

Your approach of using department-specific workspaces with their own Gold layer and connecting them to shared Silver tables using shortcuts is a solid and flexible solution. It gives each team the freedom to add their own business logic without changing the core data. This setup is actually supported in Microsoft Fabric through Lakehouse Shortcuts, which are designed exactly for sharing data between workspaces without copying it.

Your separation of environments into Dev, Test, and Prod, along with connecting them to Git branches, demonstrates good planning for long-term stability. While full Git integration in Fabric is still being developed, using repositories and deployment pipelines as a temporary solution is effective and will position you well as more automation features become available.

Your approach to access control restricting Bronze and Silver to yourself while granting departments access to Gold lakehouses and Power BI workspaces follows current data ownership practices, balancing centralized engineering with decentralized consumption. This setup supports both data quality and governance, while allowing for secure self-service analytics. The main areas to watch are the complexity from shortcuts, metadata documentation, and auditing pipeline runs. It would help to clearly document shortcut usage and use consistent naming to make dependencies easier to follow. As your platform grows, consider adding simple operational logs or dashboards to monitor data freshness and ownership.

Overall, your setup follows good design practices and matches how Microsoft Fabric is meant to be used. It puts you in a great position to grow and manage things smoothly as your data needs increase.

Hope this helps. Please feel free to reach out for further assistance.

Thank you.

Re: Workspace & Medallion Architecture Design with Centralised Logic on Bronze and Silver

ndamoi — Tue, 05 Aug 2025 09:55:09 GMT

Hi @Anonymous

Thanks so much for your quick and thoughtful response.

Your feedback is really reassuring and helps me feel more confident about the direction I’m taking. I’ll work on implementing the missing concepts and focus on documenting everything properly.

The tips you shared are super helpful and much appreciated.

Thanks again for your support!