Check your eligibility for this 50% exam voucher offer and join us for free live learning sessions to get prepared for Exam DP-700.
Get StartedDon't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.
Greetings, community. I'm continuing to explore Fabric and identify how to use the different components especially in relation to traditional ALM with Dev/Test/Production workspaces. Microsoft has talked a lot about the medallion architecture along with the benefits of OneLake and OneCopy, but I'm trying to figure out how this all aligns in terms of the ALM process in Fabric/Power BI.
Traditionally, you'd have at least three workspaces (Dev, Test, Production) with each workspace containing a version of an artifact. Production is obviously the final-state version of the artifact for consumption. You'd use a deployment pipeline to push artifacts from one stage to the next.
From a medallion-style approach that uses data warehouses, you'd:
In the world of Fabric, is the idea that all of those artifacts would live in a single workspace? So you are running everything through to from Bronze to Gold in that same workspace?
If yes, then does that mean you have to re-run everything through in Test and again in Production, which creates at least three "copies" of data between the Dev, Test, and Production workspaces? Or am I missing something?
Consider Dev/Test/Prod and Bronze/Silver/Gold as separate things. B/S/G represents transformations done on your data, whilst D/T/P are stages in safely making changes to those transformations and bringing new ones into existence.
There is going to be a degree of replication of data across all these areas. We will only replicate across D/T/P what we have responsibility to create, any data set that we only read we access the live/production data for Dev & Test. We will try to minimise the replication of our own data sets in Dev & Test (how to best to this is something we are currently investigating - poss ADF only what we need to Dev & Test)
Hello
The Dev/Test/Prod workspaces in Fabric architecture are essential to the ALM process. Fabric's medallion-style approach has the potential to combine several artifacts into a single workspace, whereas conventionally, each workspace (Dev, Test, and Production) houses a version of each artifact.
In this case, the data would move within the same workspace from Bronze to Gold. This does not, however, imply that everything in Test and Production must be redone. Because of Fabric's architecture, redundant processes can be avoided and data pipelines may be managed effectively.
Regards
Nisha Marshall
Thank you both for your replies, @Anonymous and @R1k91. That's very helpful. Two additional questions:
Trying to figure out how to work toward that OneCopy vision and ensure we don't end up just storing double/triple/quadruple the exact same data across numerous workspaces.
Hi @arpost ,
We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .
Thanks
Hi @arpost ,
Thanks for using Fabric Community.
This is an architecture question and the answer is "it depends" on requirements. Choice of architecture/layout of workspaces could be driven by cost, performance, org structure, isolation/governance boundary requirements etc.
From product perspective, Fabric is flexible and building blocks (workspaces and items) can be laid out in different ways to meet business requirements. Common patterns which we have seen thus far:
Pattern 1: Separate workspace and separate Fabric Items (WH, LH) for Bronze, Silver and Gold layers.
From ALM perspective, in the current release of Fabric, you would use separate deployment pipelines for Bronze, Silver and Gold layers which are each represented by a separate workspace.
Bronze Deployment pipeline: [WS <dev-bronze>] -> [WS <test-bronze>] -> [WS <prod-bronze>]
Silver Deployment pipeline: [WS <dev-silver>] -> [WS <test-silver>] -> [WS <prod-silver>]
Gold Deployment pipeline: [WS <dev-gold>] -> [WS <test-gold>] -> [WS <prod-gold>]
Pattern 2: Single workspace with separate LH or WH for Bronze, Silver and Gold layers.
Further reading which I would recommend to help shape your decision:
Implement medallion lakehouse architecture in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
Microsoft Fabric adoption roadmap - Power BI | Microsoft Learn
Deployment patterns for Microsoft Fabric - Azure Architecture Center | Microsoft Learn
Overview of Fabric deployment pipelines - Microsoft Fabric | Microsoft Learn
Hope this is helpful.
Hi @arpost ,
We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .
Thanks
IMHO Bronze/Silver/Gold is a transformation pattern and it's applied inside a project.
a project could be in dev, test or production environments. therefore yes you're going to copy data in multiple environments (but you could decide to have subset of data in dev and test for example).
we put all the layers inside one workspace and then we'll create a workspace per environment.
you could split layers in different workspaces but you'll need to create workspaces per environment also.
3 wrksp dev (gold siver and bronze) 3 wrksp test 3 wrksp prod = 9 wkrsp that is much more complicated to manage even from a deployment perspective.
having different environments with all the layers will duplicate data across workspaces but "one copy" principle will be true for users that should only see the production one.
moreover you can say "one copy" principle is true "at environment level" because in test environment there'll be only one gold layer, as one silver and so on.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.
User | Count |
---|---|
9 | |
6 | |
5 | |
2 | |
2 |
User | Count |
---|---|
14 | |
11 | |
7 | |
5 | |
4 |