Forum Discussion
Helper ICopyJob - add to Invoke Pipeline?
Hi fb200 ,
The roadmap does not see a rollout plan for the feature to use copy jobs (preview) in the data pipeline at the moment:
What's new and planned for Data Factory in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
You can give feedback on Home.
Data Factory Dataflow Gen2 also offers incremental importing. This feature allows you to load only new or updated data, thus increasing the efficiency of data loading. This is similar to the incremental import feature of Copy Job.
For more information and examples, please refer to the official documentation below:
Incremental refresh in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
If you have any other questions please feel free to contact me.
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
Hi fb200 ,
There is no option to use the copy job(preview) function on the data pipeline canvas.
I created a copy job(preview) in workspace and wrote its data to a warehouse, which I think is a data extraction as well as usage process, and dataflow and pipeline can be used instead.
One small note: when using multiple pipelines in one pipeline please select the “invoke pipeline(legacy)” option instead of the “invoke pipeline(preview)” option, which is currently only supported with ADF & Synapse pipelines.
Here's a screenshot of my test running in pipeline with copy job(preview) using dataflow instead.
The preview feature may have some limitations, so use an alternative if one is available.
If you have any other questions please feel free to contact me.
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!