ryomaru0825's avatar
ryomaru0825
Icon for Advocate I rankAdvocate I
1 day ago
Status:
New

Support parameterized file destination paths in Copy job

Currently, the file destination path in a Fabric Copy job is defined statically in the job configuration.

Please add support for parameterizing the destination file path, including folder and file names, so that the output location can be dynamically controlled at runtime.

For example, it should be possible to define an output path such as:

Files/output/{run_date}/{run_id}/

where run_date and run_id are supplied dynamically at runtime.

It should also be possible to pass these values from a pipeline or other orchestration process.

This would enable a single Copy job definition to be reused across different executions without creating multiple Copy jobs that differ only in their destination paths.

Use cases

  • Create execution-specific output folders using run_date and run_id
  • Dynamically partition output files by processing date
  • Prevent outputs from different executions from being mixed or overwritten
  • Pass destination path values from a pipeline or orchestration process
  • Reuse the same Copy job definition across multiple executions and environments

 

No CommentsBe the first to comment

Recent ideas