Forum Discussion
will i be able to version pipelines/dataflows
Hi we are just getting our feet wet in one new fabric ws with pipelines, dataflows, copy jobs etc targeting a medallion architecture. I think im starting to appreciate how things work in this part of the fabric space, thanks mostly to the contributors in this forum and some trial and error. im assuming i'll be chaining the pieces parts we are assembling to load bronze, silver and gold and maybe sm. the question here is will i be able to version what we are building in this ws to devops? including the "thing/arrows" that chain(s)? using the "pipeline" approach we already learned how to use for reports? if the answer is yes, im a bit concerned about the proliferation of devops projects vs enforcing unique naming conventions across dev ws's.
3 Replies
Hi db042190,
If you sync the workspace to git you can version most items. Pipelines, dataflows, and copy jobs are all supported. See the full list of supported items here:
This includes all aspects of the items, including the connections between activities in pipelines.
Hi db042190,
tayloramy is right on the versioning side. Pipelines, Dataflow Gen2 and Copy Jobs are all currently supported by Fabric Git integration, so the item definition itself is versioned, including the pipeline activity structure and dependencies between those activities.
On your Azure DevOps proliferation concern, I would not create a separate DevOps project for every Fabric workspace.
A Fabric workspace can only be connected to one Git branch at a time, so Microsoft's current development guidance is to give developers an isolated environment and have each developer work against their own branch. Those branches can still live within the same Azure DevOps project/repository.
I would normally organise it around the solution or data domain instead: one repo, a shared main branch, developer/feature branches, and the corresponding Fabric development workspaces. Microsoft also notes that a developer can reuse the same private workspace for later branches rather than creating a new workspace for every piece of work.
The other thing I would plan early is environment configuration. Git handles the item definitions, but dev/test/prod-specific values and connections should be treated separately. Variable libraries are useful for keeping those values out of hard-coded pipeline definitions.
Then once a change is reviewed and merged, you can use the appropriate Fabric CI/CD approach to promote it into test and production.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
