topic Re: How to manage data pipeline deployments across different environments ? in Pipelines

How to manage data pipeline deployments across different environments ?

alfBI — Wed, 07 Feb 2024 08:19:41 GMT

Hi,

We are working with different workspaces to manage TST and PRO environments. We use to use deployment pipelines to move objects across them, but unfortanetly data pipelines are not yet supported...

https://learn.microsoft.com/en-us/fabric/cicd/deployment-pipelines/understand-the-deployment-process

I have been checking on data factory incoming changes but I don't see any change on this area.

https://learn.microsoft.com/en-us/fabric/release-plan/data-factory

Question is: For that people curring using different environments and data pipelines, How you move them across the workspaces? I can not imagine having to recreate them manually... Does it exist some workaround (export definition to JSON or somethin similar) to move them across?

Regards

Alfons

Re: How to manage data pipeline deployments across different environments ?

NandanHegde — Wed, 07 Feb 2024 09:16:45 GMT

Unfortunately manual recreation is the only way around as of today because the JSON are non editable and read only.

And also Git integration is currently not supprted.

https://learn.microsoft.com/en-us/fabric/release-plan/data-factory#git-pipelines

But as per roadmap, Git integration is there which should be a sign towards CICD as well

Re: How to manage data pipeline deployments across different environments ?

Anonymous — Wed, 07 Feb 2024 09:25:50 GMT

Hi alfBI ,

Thanks for using Fabric Community.
As NandanHegde said currently data pipelines is not supported in Fabric Deployment.

We have an update from internal team -
" Data pipelines will soon start a private preview for deployment pipelines. If you have a user who wish to join, please send an email to fabricdfcicd[at]Microsoft[dot]com "

Hope this is helpful. Please let me know incase of further queries

Re: How to manage data pipeline deployments across different environments ?

alfBI — Wed, 07 Feb 2024 09:33:15 GMT

Yes, definitely. It helps. I am curious about the internal team statement you show.
Which is the way to participate on this private previews? (in this case it might also useful to us)

Many Thx

Re: How to manage data pipeline deployments across different environments ?

Anonymous — Wed, 07 Feb 2024 09:35:15 GMT

Hi alfBI ,

Please send an email to fabricdfcicd[at]Microsoft[dot]com.
Our team will coordinate with you and help you with the same.

Hope this helps. Please let us know if you have any other queries.

Re: How to manage data pipeline deployments across different environments ?

NandanHegde — Wed, 07 Feb 2024 09:36:10 GMT

Note : Since it is a private preview you would have to have some NDA clause signed and you cannot disclose it to others 🙂

Re: How to manage data pipeline deployments across different environments ?

Anonymous — Thu, 08 Feb 2024 10:59:28 GMT

Hi alfBI ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .