Forum Discussion

seihl's avatar
seihl
Frequent Visitor
1 month ago
Solved

CI/CD deployment for arbitrary workspaces without Fabric Git Integration

I'm trying to understand the recommended/official approach for CI/CD of Microsoft Fabric Apps in an enterprise environment where Fabric Git Integration is not an option.

Let's say we develop reusable Fabric solutions that must be deployed to many customer workspaces, and we cannot use Fabric Git Integration because:

  • Fabric does not have access to our GitHub Enterprise (GHE) instance.
  • Even if it did, Fabric Git Integration seems to be based on a workspace ↔ branch association, whereas our deployment model is any branch β†’ any workspace depending on the release pipeline.
  • As an ISV we want local-first, code-first, custom branching strategy (PRs, etc.) and complete deployment automation control.


In other words, we need a traditional CI/CD approach where the deployment pipeline decides which artifact version (branch/tag/commit) is deployed to which workspace, and just treat Fabric as a deployment runtime.

This seems doable for Notebook, Spark Jobs Definitions and Airflow DAGs, but not really for the rest of Fabric workloads.


Take Eventstream for instance: using the Fabric VS Code extension (or the REST API), we can retrieve the Eventstream definition as JSON. However, the exported definition contains many environment-specific identifiers, for example:

{
  "namespaceResourceId": "/subscriptions/.../providers/Microsoft.EventGrid/namespaces/...",
  "workspaceId": "a373059b-e0ea-48da-811f-8c324600a4f6",
  "itemId": "6247h8d1-d889-47a8-b096-d2114809gd27"
}


These IDs are specific to the source environment and cannot be reused directly when deploying to another customer workspace.

I'm trying to understand whether we're expected to build our own templating/parameterization layer around these definition JSON files (i.e., fabric-cicd + parameters.yml) , or if there is another recommended approach.

  • Hi User,

    Your understanding is largely correct. Today, Fabric Git Integration is the recommended experience for source control and CI/CD, but it is primarily designed around a workspace-to-branch model. For ISV scenarios requiring any branch β†’ any workspace deployments and GitHub Enterprise environments that Fabric cannot access, there isn't currently a fully equivalent "code-first" deployment model for all Fabric artifact types.

    For enterprise CI/CD without Fabric Git Integration, a common approach is:

    • Store Fabric artifacts in Git using the Fabric VS Code extension, the Fabric REST APIs, or supported project formats where available.

    • Use your existing CI/CD platform (e.g., GitHub Actions, Azure DevOps, Jenkins) to determine which branch, tag, or commit should be deployed.

    • Parameterize environment-specific values (workspace IDs, connection strings, Eventstream resource IDs, etc.) during deployment rather than committing them directly.

    • For artifacts such as Eventstream that contain environment-specific identifiers, maintaining a templating or parameterization layer (for example, using fabric-cicd with environment-specific parameter files) is currently one of the most practical approaches.

    • Where supported, leverage deployment pipelines and REST APIs to automate artifact promotion, while recognizing that not every Fabric workload currently offers the same level of deployment portability.

    At present, there isn't a built-in mechanism that automatically remaps all environment-specific IDs for every Fabric artifact type during deployment. This is an area where many customers and ISVs have requested improvements.

    For more information:

    Could you also clarify:

    • Are you targeting all Fabric artifact types, or primarily Data Factory, Eventstream, and Real-Time Intelligence artifacts?

    • Are your customer workspaces located in different tenants, or within a single tenant?

    • Have you evaluated the open-source fabric-cicd tooling, and if so, are there specific artifact types where you've found its parameterization insufficient?

    These details would help determine whether there's an alternative deployment pattern that better fits your architecture.

     

    πŸ’‘ Helpful? Give a Kudos πŸ‘ β€” keep the community growing.

    βœ… Solved your issue? Mark this as the Accepted Solution βœ”οΈ

    Best regards,
    Prince Singh | Data Science & Microsoft Fabric Enthusiast

8 Replies

  • Hi User,

    Your understanding is largely correct. Today, Fabric Git Integration is the recommended experience for source control and CI/CD, but it is primarily designed around a workspace-to-branch model. For ISV scenarios requiring any branch β†’ any workspace deployments and GitHub Enterprise environments that Fabric cannot access, there isn't currently a fully equivalent "code-first" deployment model for all Fabric artifact types.

    For enterprise CI/CD without Fabric Git Integration, a common approach is:

    • Store Fabric artifacts in Git using the Fabric VS Code extension, the Fabric REST APIs, or supported project formats where available.

    • Use your existing CI/CD platform (e.g., GitHub Actions, Azure DevOps, Jenkins) to determine which branch, tag, or commit should be deployed.

    • Parameterize environment-specific values (workspace IDs, connection strings, Eventstream resource IDs, etc.) during deployment rather than committing them directly.

    • For artifacts such as Eventstream that contain environment-specific identifiers, maintaining a templating or parameterization layer (for example, using fabric-cicd with environment-specific parameter files) is currently one of the most practical approaches.

    • Where supported, leverage deployment pipelines and REST APIs to automate artifact promotion, while recognizing that not every Fabric workload currently offers the same level of deployment portability.

    At present, there isn't a built-in mechanism that automatically remaps all environment-specific IDs for every Fabric artifact type during deployment. This is an area where many customers and ISVs have requested improvements.

    For more information:

    Could you also clarify:

    • Are you targeting all Fabric artifact types, or primarily Data Factory, Eventstream, and Real-Time Intelligence artifacts?

    • Are your customer workspaces located in different tenants, or within a single tenant?

    • Have you evaluated the open-source fabric-cicd tooling, and if so, are there specific artifact types where you've found its parameterization insufficient?

    These details would help determine whether there's an alternative deployment pattern that better fits your architecture.

     

    πŸ’‘ Helpful? Give a Kudos πŸ‘ β€” keep the community growing.

    βœ… Solved your issue? Mark this as the Accepted Solution βœ”οΈ

    Best regards,
    Prince Singh | Data Science & Microsoft Fabric Enthusiast

    • seihl's avatar
      seihl
      Frequent Visitor

      Even a typical fabric-cicd setup seems to require connecting your Fabric workspace to a Git repo at least once to generate the artifact files and folder structure the library expects, unless you recreate them yourself (for example, via the REST APIs or the VS Code extension).

      It is also not without its limitations: fabric-cicd doesn't support Workspace settings (i.e., Managed Private Endpoints or Fabric Connections), so you need additional tooling, such as the Terraform Fabric provider, to get everything wired up.

      But, as far as I can tell, that's the only realistic option right now.

      • v-abhinavmu's avatar
        v-abhinavmu
        Community Support

        Hi seihl,

         

        Thanks for the clarification and for sharing your insights. I appreciate you taking the time to explain the current limitations and the available options. This is very helpful and provides good context for anyone exploring similar deployment scenarios.

         

        Thank you.

  • Hi seihl ,

     

    In the first point are you saying Fabric doesnt support GitHub Enterprise?

    You can associate a branch to one workspace only like one-one. 

    In a traditional way, we use separate branch for all environments. Similarly here you can have branches and workspaces. If you want to setup approval gates, it is possible with Azure Devops.

    Please refer these article it is very well explained with detailed steps.

    https://learn.microsoft.com/en-us/fabric/cicd/tutorial-fabric-cicd-azure-devops

     

     

    • seihl's avatar
      seihl
      Frequent Visitor
      Sorry, I should've been more precise: Fabric doesn't support direct Git integration with privately hosted GHES.
  • You should be able to use all the supported items together and then deploy them collectively from the Git source. Which can be done in anumber of ways, including by merging branches all using all of the items in the branch as a source to be deployed through API's or a layer of tooling such as the fabric-cicd libary.

     

    I assume you are using GitHub Enterprise Server?