Dataflows are supported by the Deployment pipelines, but there is no Git integration for dataflows.
With the introduction of Git integration of workspaces, it would be fantastic if Dataflows could be stored and traced and reviewed in Git same like it is possible now for datasets and reports.
20 Comments
It is a good idea.
Datadflows are all edited online and without saved files on local machines. Traceability is so lacking here.
- Our team is aware of the need and we do plan to support Dataflow Gen2 in the future with Git integration. We will keep you posted once timelines are available.
Hoping we get an update on this soon!
Please don't hold this back in favor of making everything available! :) We need this Now!
Our need is for Gen1 dataflows. Some way to quickly check in what is currently a single .json file that can be easily formatted and compared.
We do this with a series of PowerShell scripts today. But git integration has great value in letting us know when something isn't up to date.
Please don't delay the easier task in favor of pushing the Fabric SKU.
Currently we can't use GIT solution, as there will be error when there's a dataflow in one workspace, and you have a repository for this workspace, but then you want to use that repository to connect to another empty workspace, there will be an error when synch, so please add this feature ASAP.
dbeavonAdvocate I
Any updates here? Is this going to take months or years?
How can you guys make call this stuff "GA" without any sort of CI/CD or git integration? You guys need to eat your own dogfood. It is shameful that you build this platform for hosting software solutions, yet have no meaningful plan for supporting CI/CD or source control. In past decades, Microsoft used to build very high quality development tooling for data engineers. Now I just want to avoid anything from Microsoft, and seek open-source alternatives which are much more robust, have better tooling, and have better support.
This is horrible .. also it should be possible to adjust the JSON file especially for migration purposes of other platforms like Synapse.
Currently every single dataflow has to be set up manually and connected to a destination.
You should be able to automate this process using the JSON code.
Definitely it's a great addition to have Git integration for dataflow. Data flow is going to be a greate ETL tool in Fabric and needs to manage the versions of the changes.
Any status update?
