Forum Discussion
Gen2 dataflows and deployment pipelines
- 1 year ago
Hi mikesmall
As of now, Gen2 dataflows (Dataflows Gen2) are not yet fully supported in Power BI Deployment Pipelines, which is likely why you're not seeing them show up in your pipeline environments during testing. Deployment pipelines currently focus on semantic models, reports, and dashboards, and do not automatically include Gen2 dataflows—especially those not tied to a specific destination like a Lakehouse or Warehouse. Since Gen2 dataflows rely on Microsoft Fabric's OneLake infrastructure, their integration with deployment pipelines is still evolving and subject to future roadmap updates.
Furthermore, Gen2 dataflows often require a staging destination (usually a Lakehouse or Warehouse) to properly materialize and store their output, which is not just a convenience—it's essential for how they function and persist data. If you left the destination unset for simplicity, the dataflow wouldn't create or associate with any deployable artifact (like a Lakehouse table), meaning there's nothing concrete for the pipeline to pick up or compare across environments.
In practice, if you're testing Gen2 dataflows and want to prepare for future deployment pipeline integration, it's recommended to assign an explicit destination (e.g., a Lakehouse in your Fabric workspace). Also, monitor the Fabric release roadmap or official Microsoft documentation closely, as support for deploying Gen2 dataflows through pipelines is a highly requested feature and may be included in future updates.
In the meantime, deployment of Gen2 dataflows across environments typically needs to be done manually (e.g., exporting and importing definitions), or automated using APIs or Fabric CLI once such tools support the feature.
Good news. Dataflows Gen2 with CI/CD (preview) are supported in a deployment pipeline.
"Dataflow Gen2 now supports Continuous Integration/Continuous Deployment (CI/CD) and Git integration [...] Additionally, you can use the deployment pipelines feature to automate the deployment of dataflows from your workspace to other workspaces. [...]"
See Dataflow Gen2 with CI/CD and Git integration - Microsoft Fabric | Microsoft Learn
- raschaoot1 year agoFrequent Visitor
Just a shame you can't refresh these CI/CD flows yet from a pipeline... pretty limited when you have two dozen flows with all kinds of dependencies (for which the pipeline is very neat).
- Christophe931001 year agoFrequent Visitor
Hi
Are there deployments pipeline stages ( test / prod) rules to modify source or target connections and to modify dfgen2 parameters ?
--
How about :
- copy job change
-Pbi paginated reports sourcing change
- shortcuts target change
on deployments pipelines ?
--
Data factory pipeline can use the new librairies variable (on preview)