Forum Discussion

reddyr2502's avatar
reddyr2502
Regular Visitor
22 days ago
Solved

Folders Structures in workspace

Hi, I am the only person managing data analytics at a startup. We currently use a single Microsoft Fabric workspace for our Bronze, Silver, and Gold data layers. What is the best way to organize an...
data engineering
  • Murtaza_Ghafoor's avatar
    Murtaza_Ghafoor
    13 days ago

    HI reddyr2502​ ,

    It depends on your organization size, for example if your organization is small and there is less potential of future growth, then it is recommended that keep all artifacts in single workspace.

    However, i would strongly recommend creating separate workspaces as mentioned under:

                                 
                                  WS_Dev ----> WS_UAT ----> WS_Prod

    Now in Dev Workspace  you keep all artifacts in the folders like for an example for Lakehouse , use  Lakehouse and for notebooks use Notebooks and so on. 

    Deployment Pipeline:

    Now everything that should move from Dev to UAT & UAT to PROD must be through Deployment Pipelines, a built in feature in fabric premium portal. Thanks 






    Reference: Microsoft


      If this helps, ✓ Mark as Kudos | Help Other



tayloramy's avatar
tayloramy
Icon for Super User rankSuper User
21 days ago

Hi reddyr2502​,
It really depends on how you work and how you've structured your environment.

I've seen one folder per source system before, and then inside that are subfolders for bronze, silver, and gold components. 

I've also seen the root folders be bronze, silver, and gold, and then in each of those are subfolders for each system. 

My environment is metadata driven for bronze and silver layers, so there's nothing system specific at that level, I have a folder for some shared functions that I've abstracted away from the main notebooks so they can be reused across layers, and then I have a single notebook for bronze and a single notebook for silver that will read from the metadata tables and load what needs to be loaded. 

Then at my gold layer I have a top level folder for gold, and inside I have a folder for each resulting data product.