Folder structure and ADO/Fabric sync

[Apelsinas]

Hello everyone,

I’m working on setting up a data infrastructure within Microsoft Fabric, and I’m currently using Azure DevOps for source control and CI/CD. My goal is to create a highly organized and scalable workflow for handling our environment’s Bronze, Silver, and Gold layers with clear separation for development, testing, and production.

Here is our current structure:

• Azure DevOps: We use a single repository with a folder structure to organize different workspaces and environments.
• Fabric Workspaces: We have three separate layers (Bronze, Silver, Gold), each with distinct Dev, Test, and Prod environments, resulting in nine Fabric workspaces:
  • Bronze-Dev, Bronze-Test, Bronze-Prod
  • Silver-Dev, Silver-Test, Silver-Prod
  • Gold-Dev, Gold-Test, Gold-Prod

I have a few specific questions:

1. Folder Structure Management: What folder structures are others using in Azure DevOps to manage multiple Fabric environments? Any best practices for handling complexity, especially across environments and data layers?

2. Workspace Synchronization & Sync Issues: Currently, only our Dev workspace is directly synced with Git, and we use Fabric’s deployment pipelines to promote content from Dev to Test, and then to Prod. However, maintaining consistent environments has challenges, especially with versioning and manual sync issues. Are there approaches or tools that help make this process smoother?

3. Microsoft’s Roadmap & Improvements: Is Microsoft actively working on improvements to facilitate environment syncing across Fabric and DevOps? Any upcoming features or known best practices that could make multi-environment management easier?

Thanks in advance for any insights or recommendations!

[spencer_sa · Accepted solution]

We have a similar workspace structure (except we have separated code into its own workspace)

We have folders in the DevOps repo that match the layers and the process/code workspace.  Lakehouses don't generate much in the way of artifacts - just a stub entry really.

The main branch is connected to the dev workspaces.

We've declined (for now) to use the Deployment Pipelines and are instead going to use two branches - test and release - to connect to our test and prod environments.

The biggest issue you'll find in general is trying to rewrite the default lakehouses of any artifacts.  Yes deployment rules do work, but you have to currently maintain them separately.  This is likely to change in the future with some of the stuff on the roadmap.  We are currently looking at a couple of ways around this involving absolute lakehouse paths and API calls to change the default lakehouses.

In terms of synching branches to main, it's possible to set up DevOps scripts that are automatically triggered on pull requests to perform a synch (and/or trigger semantic model refreshes) once the pull request has executed.

There are some things in the roadmap that will make things easier for deployment pipelines (such as parameters)