topic Re: Medallion architecture with CI/CD Pipelines in Data Engineering

Medallion architecture with CI/CD Pipelines

nomad_ch — Tue, 21 Oct 2025 13:40:10 GMT

Hello,

I’m an analyst transitioning to data engineering and I am quite new to Fabric. I am learning how to structure a medallion architecture (Bronze → Silver → Gold) in Microsoft Fabric across Dev, Test, and Prod environments.

I want to set up my OneLake and warehouse/lakehouse correctly before building pipelines. I’m thinking of this structure:

- OneLake (staging area for raw files, e.g., Files/raw/)

- Three workspaces for environments:

- Dev_Workspace -> Warehouse or Lakehouse with BronzeSchema, SilverSchema, GoldSchema

- Test_Workspace -> Warehouse or Lakehouse with BronzeSchema, SilverSchema, GoldSchema

- Prod_Workspace -> Warehouse or Lakehouse with BronzeSchema, SilverSchema, GoldSchema

Does this structure make sense for Fabric?

Is Bronze needed at all in Dev/Test for testing ingestion from OneLake?

Any tips for CI/CD with this setup?

Thanks!

Re: Medallion architecture with CI/CD Pipelines

tayloramy — Tue, 21 Oct 2025 13:55:51 GMT

Hi @nomad_ch,

This setup makes sense to me.

I do a silimar thing, using a Lakehouse as my bronze and silver layers, and then I use a warehouse as my gold layer so I can apply RLS to it.

as for CI/CD, what I recommend is connecting GitHub or Azure DevOps to your dev workspace for version control, and then use deployment pipelines to move items from dev up to test and then to prod.

There's an accelerator in the Fabric Toolbox for this: fabric-toolbox/accelerators/CICD/Deploy-using-Fabric-deployment-pipelines at main · microsoft/fabric-toolbox

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

Re: Medallion architecture with CI/CD Pipelines

nielsvdc — Tue, 21 Oct 2025 20:07:58 GMT

Hi @nomad_ch,

If you are just starting out and want to keep it simpel and small, 3 workspaces is good enough. Depending on the processing workload on dev-test and production, you might also want to think about a production capacity and a dev-test capacity, so that any development and test processes won't burn out your ability to run production due to CU shortage.

Instead of developing a framework yourself, you can also check the ready-to-use metadata driven framework from Erwin de Kreuk on Github https://github.com/edkreuk/FMD_FRAMEWORK/.

Hope this helps. If so, please give a Kudos 👍 and mark as Accepted Solution ✔️.

Re: Medallion architecture with CI/CD Pipelines

v-dineshya — Wed, 22 Oct 2025 11:28:08 GMT

Hi @nomad_ch ,

Thank you for reaching out to the Microsoft Community Forum.

Hi @tayloramy and @nielsvdc , Thank you for your prompt responses.

Hi @nomad_ch , Could you please try the proposed solutions shared by @tayloramy and @nielsvdc ? Let us know if you’re still facing the same issue we’ll be happy to assist you further.

Regards,

Dinesh

Re: Medallion architecture with CI/CD Pipelines

KevinChant — Wed, 22 Oct 2025 16:03:42 GMT

This looks fine to me. in addition to the previous framework shared you can also look to use Fabric Accelerator which roots can be traced back to offerings for other services:

https://github.com/bennyaustin/fabric-accelerator

For CI/CD across traditional DTAP environments you might also want to look into using the fabric-cicd Python library with your ALM service of choice.
https://microsoft.github.io/fabric-cicd/latest/

I have a fair few posts relating to performing CI/CD in Fabric which might help:
https://www.kevinrchant.com/?s=fabric+ci%2Fcd

Re: Medallion architecture with CI/CD Pipelines

v-dineshya — Mon, 27 Oct 2025 18:14:40 GMT

Hi @nomad_ch ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

Regards,

Dinesh