Forum Discussion
Medallion architecture with CI/CD Pipelines
Hi nomad_ch,
This setup makes sense to me.
I do a silimar thing, using a Lakehouse as my bronze and silver layers, and then I use a warehouse as my gold layer so I can apply RLS to it.
as for CI/CD, what I recommend is connecting GitHub or Azure DevOps to your dev workspace for version control, and then use deployment pipelines to move items from dev up to test and then to prod.
There's an accelerator in the Fabric Toolbox for this: fabric-toolbox/accelerators/CICD/Deploy-using-Fabric-deployment-pipelines at main · microsoft/fabric-toolbox
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
This looks fine to me. in addition to the previous framework shared you can also look to use Fabric Accelerator which roots can be traced back to offerings for other services:
https://github.com/bennyaustin/fabric-accelerator
For CI/CD across traditional DTAP environments you might also want to look into using the fabric-cicd Python library with your ALM service of choice.
https://microsoft.github.io/fabric-cicd/latest/
I have a fair few posts relating to performing CI/CD in Fabric which might help:
https://www.kevinrchant.com/?s=fabric+ci%2Fcd
Super UserHi nomad_ch,
If you are just starting out and want to keep it simpel and small, 3 workspaces is good enough. Depending on the processing workload on dev-test and production, you might also want to think about a production capacity and a dev-test capacity, so that any development and test processes won't burn out your ability to run production due to CU shortage.
Instead of developing a framework yourself, you can also check the ready-to-use metadata driven framework from Erwin de Kreuk on Github https://github.com/edkreuk/FMD_FRAMEWORK/.
Hope this helps. If so, please give a Kudos 👍 and mark as Accepted Solution ✔️.