Forum Discussion

AJAJ's avatar
AJAJ
Helper IV
7 months ago
Solved

Copy Activity Dynamic Source Connection

Hi Currently my pipelines on dev and prod have the same source. So basically my ingestion pipeline + EDW pipeline run daily. Reads same source twice. By prod and then by Dev. In order to avoid doub...
data engineering
Data Pipeline
  • v-menakakota's avatar
    v-menakakota
    7 months ago

    Hi AJAJ ,

    I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you .

     

     

    Best Regards, 
    Community Support Team

deborshi_nag's avatar
deborshi_nag
Super User
7 months ago

Hello  AJAJ 

 

If your Dev/Prod are in same server with different DB/table, you can keep a single Linked Service and parameterize the dataset paths/table names:
 
Dataset parameters:

{
  "env": "prod|dev",
  "system": "crm",
  "entity": "customer",
  "snapshotDate": "2025-12-27",
  "runId": "20251227T010003Z"
}
 
File/folder path expression:

@format('/raw/{0}/{1}/{2}/ingest_date={3}/run_id={4}',
        dataset().env,
        dataset().system,
        dataset().entity,
        dataset().snapshotDate,
        dataset().runId)
 
Table naming (if using SQL landing):
@concat(dataset().env, '_', dataset().system, '_', dataset().entity)
 
This doesn’t switch credentials, only the logical location.
 
KIndly accept this as a solution or a kudos if that helps! 

 

 

  • v-menakakota's avatar
    v-menakakota
    Community Support
    7 months ago

    Hi AJAJ ,

    I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you .

     

     

    Best Regards, 
    Community Support Team