Copy Activity Dynamic Source Connection

[AJAJ]

Hi

Currently my pipelines on dev and prod have the same source. So basically my ingestion pipeline + EDW pipeline run daily. Reads same source twice. By prod and then by Dev.

In order to avoid double read of source twice one by dev and one by prod,

Is there a way to dynamically point sources based on flag on copy activity?

Below is diagram of existing set up and the 2nd part is thinking to break apart ingestion pipeline and dynamically decide where to pick up data from.

If not, is there any other way to achieve this?

Is there a better way to do this in fabric?

Activities used in pipeline.

Look up activity to fetch tablenames from reference table, - foreach loop with copy activity inside populating staging tables. 

[v-menakakota]

Hi @AJAJ ,
Thanks for reaching out to the Microsoft fabric community forum. 

At the moment, Fabric pipelines don’t support dynamically switching the source system of a Copy activity at runtime based on a flag. While parameters exist, using them to change sources like this isn’t a supported or reliable pattern.

The real issue is that both Dev and Prod are reading from the source systems. In Fabric, the recommended approach is to read from the source only once, usually in Prod, land the data into Raw/Staging, and then let both Dev and Prod pipelines read from that staged data.

If Dev needs to test transformations, it should use the data already available in Raw/Staging (or a subset of it), instead of reading from the source again. This avoids double reads, reduces load on source systems, and keeps the setup simpler and more stable.

Best Regards, 
Community Support Team