Dataflow Gen 2 vs Dataflow Gen2 (CI/CD)

[tan_thiamhuat]

can I know the main differences between the above 2?

[burakkaragoz · Accepted solution]

Hi @tan_thiamhuat ,

Just to add a bit more on top of what Cooksalador already shared — the CI/CD version of Dataflow Gen2 is really aimed at teams who want to treat their data pipelines like software projects. So if you're working in an environment where version control, automated testing, and deployment pipelines are a must, then the CI/CD flavor is what you’d go for.

One thing I noticed when testing both is that the CI/CD version gives you more control over lifecycle management. You can push changes through environments (dev, test, prod) using Git branches, which is super helpful for governance and rollback scenarios.

But yeah, if you're just building flows for ad-hoc or internal use without a full devops setup, the regular Gen2 might be simpler and faster to get going with.

Hope that helps a bit more.

If my response resolved your query, kindly mark it as the Accepted Solution to assist others. Additionally, I would be grateful for a 'Kudos' if you found my response helpful.

[Cookistador]

Hi @tan_thiamhuat 

Dataflow Gen2 (CI/CD) is a version of Dataflow Gen2 integrated with CI/CD and Git tools, offering improved version management and automated deployment. The standard version of Dataflow Gen2 also lets you create data flows, but lacks CI/CD functionalities for collaborative management and deployment.

The separation of Dataflow Gen2 and Dataflow Gen2 (CI/CD, Preview) is supposedly because the CI/CD and Git integration features are still in beta, so they are not yet fully integrated into the standard release.

You can find more about Dataflow Gen2 (CI/CD) under the following link

Dataflow Gen2 with CI/CD and Git integration - Microsoft Fabric | Microsoft Learn