Forum Discussion
Dataflow gen2 in deployment pipelines
The dataflow gen2 has been listed as preview feature in the deployment pipeline.
however, I don't see its still compatible to be moved in a deployment pipeline.
Anyone knows when this will be available?
Your feedback is extremely valuable and highlights a key area for improvement. as Microsoft's official support for for pipeline orchestration support with CI/CD-enabled Dataflow Gen2 is limited we recommend to submit your ideas here to Fabric team: https://ideas.fabric.microsoft.com/ideas/search-ideas/
as feedback given here will closely monitored by PG team. We are committed to ensuring our solutions meet your business needs and workflows effectively. Thank you for your patience and understanding as we work to enhance this functionality.
18 Replies
Hello,
Will this limitation be removed in a future release? This is a major problem because the main purpose of the dataflows is to transform data in pipeline orchestration.
- Orchestrating a refresh of a Dataflow Gen2 with CI/CD and Git support isn't possible in Fabric data pipelines.
Please vote this idea here
This seems to have been addressed in a recent announcement. We can save the dataflow ge2 as dataflow (CI/CD) - that will allow us to integrate them with GIT and also invoke from the pipeline.
Fabric March 2025 Feature Summary | Microsoft Fabric Blog | Microsoft Fabric
Gpop13, Thanks for reaching out to MS Fabric community support.
as per docs the release of CI/CD and Git integration support has been rescheduled to January to ensure the quality of the release and ensure that there is no impact to existing operations. We apologize for any inconvenience and appreciate your understanding.
I’ll keep a close watch for updates regarding its integration and will notify you as soon as I learn more, and I recommend keeping an eye on the official Microsoft documentation, as it is regularly updated with the latest information and feature availability
please refer to below similar reasolved thread from community and let me know if this help you
Solved: What's up with Dataflow Gen2 and Dataflow Gen2 (CI... - Microsoft Fabric Community
Thanks,
Prashanth Are
MS Fabric community support
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query
Thank you for the response. We are already at the end of January, hope to get the release/update in a couple of days now.
Also, I do not see the option below too
Once the CICD is enabled, and once we are able to use the dataflow gen2 in a deployment pipeline, I hope that it will support existing dataflow gen2 as well, and not just the ones newly created with this checkbox?
Gpop13, Thanks for your valuable insights and feedback.
I will closely monitor updates on its integration and inform you as soon as I receive any new information. meanwhile, I request you to submit your feedback here in documentation. so, that product team will take appropriate action here. this will help community members who are dependent on this usecase.
Thanks,
Prashanth Are
MS Fabric community support
please Mark post as a solution, this will help others! If our response(s) assisted you in any way, don't forget to drop me a "Kudos".
Please vote this idea here to support
Orchestrating a refresh of a Dataflow Gen2 with CI/CD and Git should be supported in Fabric data pipelines.
I am hitting the same issue. How are people actually deploying DFG2 resources to multi-stage environments via pipelines currently? Or is it just that nobody is doing it because it's not possible?
Hi Stinkys - The DFG2 are still not supported in depl pipelines. But DFG2 CICD versions are.
I did try that approach but the biggest issue I faced was even if the DFG2 is deployed to next stage, all the destination settings point to the source stage. Example, if in a DFG2, Query1 writes to a lakehouse table in dev env, after its deployed to Test, this setting will still be the same and NOT pointing to corressponding Test lakehouse.
This was a very manually work for me.
In the first go, I recreated all the DFG2 in all stages, updated the target connections accordingly. Now whener there is a change to existing query, I manually update (advanced query) in each env (I know this is not the best solution but closest to an easier approach I could have)
We are following up once again regarding your query. Could you please confirm whether you have raised ideas forum with Microsoft?
If the issue has been resolved, we kindly request you to share the resolution or key insights here to help others in the community. If we don’t hear back, we’ll go ahead and close this thread.
Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.
Thank you for your understanding and participation.
