topic Parametrize the Copy Activity in Data Pipeline for multiple environments in Pipelines

Parametrize the Copy Activity in Data Pipeline for multiple environments

williamgraver — Fri, 26 Jul 2024 21:03:16 GMT

Hi Community!

I'm working on a project where we have a deployment pipeline for multiple environments (dev, staging, and prod) and so different connections/connection strings.

We are leveraging the Copy Activity and the connections through On-premise Data Gateway. Unfortunately there is no way that I found to parametrize the connections in the Copy Activity depending on the workspace (and so the env).

I heard about this https://learn.microsoft.com/en-us/fabric/release-plan/data-factory#enabling-customers-parameterize-connections but Q4 is far from now, it seems impossible to have a secure pipeline in current conditions...

Any recommendations in that regards?

Thanks in advance.

Re: Parametrize the Copy Activity in Data Pipeline for multiple environments

Anonymous — Mon, 29 Jul 2024 02:10:07 GMT

Hi williamgraver ,

Managing different connections and connection strings across multiple environments in Azure Data Factory can be streamlined using continuous integration and delivery (CI/CD) practices. You can look at this document: Continuous integration and delivery - Azure Data Factory | Microsoft Learn

You can also use git integration and deployment for data pipelines. You can look at this document: Git integration and deployment for data pipelines - Microsoft Fabric | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.