Forum Discussion

fb200's avatar
fb200
Helper I
1 year ago
Solved

CopyJob - add to Invoke Pipeline?

I want orchestrate all my pipelines (pl1 ->pl2 -> cj - pl4)> with invoke pipelines. I have not found a way to add copyjob. Anyone has a solution?

data engineering
data pipeline

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    Hi fb200 ,

     

    There is no option to use the copy job(preview) function on the data pipeline canvas.

     

    I created a copy job(preview) in workspace and wrote its data to a warehouse, which I think is a data extraction as well as usage process, and dataflow and pipeline can be used instead.

     

    One small note: when using multiple pipelines in one pipeline please select the “invoke pipeline(legacy)” option instead of the “invoke pipeline(preview)” option, which is currently only supported with ADF & Synapse pipelines.

     

    Here's a screenshot of my test running in pipeline with copy job(preview) using dataflow instead.

     

    The preview feature may have some limitations, so use an alternative if one is available.

     

    If you have any other questions please feel free to contact me.

     

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

  • fb200's avatar
    fb200
    Helper I
    1 year ago

    Hi Yang,

    thanks for your feedback and thanks for the workaround.

    I use copyjob with incremental load. And I like the no-code capability to append the tables.

    I believe its a big benefit to add new data to exisiting tables. 

     

    Is there any plan on the roadmap to add this feature to the invoke pipelines?