Forum Discussion

db042190's avatar
db042190
Icon for Impactful Individual rankImpactful Individual
18 days ago
Solved

will i be able to version pipelines/dataflows

Hi we are just getting our feet wet in one new fabric ws with pipelines, dataflows, copy jobs etc targeting a medallion architecture.   I think im starting to appreciate how things work in this part ...
  • ShivekMaharaj's avatar
    17 days ago

    Hi db042190​,

    tayloramy​ is right on the versioning side. Pipelines, Dataflow Gen2 and Copy Jobs are all currently supported by Fabric Git integration, so the item definition itself is versioned, including the pipeline activity structure and dependencies between those activities.

    On your Azure DevOps proliferation concern, I would not create a separate DevOps project for every Fabric workspace.

    A Fabric workspace can only be connected to one Git branch at a time, so Microsoft's current development guidance is to give developers an isolated environment and have each developer work against their own branch. Those branches can still live within the same Azure DevOps project/repository.

    I would normally organise it around the solution or data domain instead: one repo, a shared main branch, developer/feature branches, and the corresponding Fabric development workspaces. Microsoft also notes that a developer can reuse the same private workspace for later branches rather than creating a new workspace for every piece of work.

    The other thing I would plan early is environment configuration. Git handles the item definitions, but dev/test/prod-specific values and connections should be treated separately. Variable libraries are useful for keeping those values out of hard-coded pipeline definitions.

    Then once a change is reviewed and merged, you can use the appropriate Fabric CI/CD approach to promote it into test and production.

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.