- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
CopyJob - add to Invoke Pipeline?
I want orchestrate all my pipelines (pl1 ->pl2 -> cj - pl4)> with invoke pipelines. I have not found a way to add copyjob. Anyone has a solution?
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @fb200 ,
The roadmap does not see a rollout plan for the feature to use copy jobs (preview) in the data pipeline at the moment:
What's new and planned for Data Factory in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
You can give feedback on Home.
Data Factory Dataflow Gen2 also offers incremental importing. This feature allows you to load only new or updated data, thus increasing the efficiency of data loading. This is similar to the incremental import feature of Copy Job.
For more information and examples, please refer to the official documentation below:
Incremental refresh in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
If you have any other questions please feel free to contact me.
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Yang,
thanks for your feedback and thanks for the workaround.
I use copyjob with incremental load. And I like the no-code capability to append the tables.
I believe its a big benefit to add new data to exisiting tables.
Is there any plan on the roadmap to add this feature to the invoke pipelines?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @fb200 ,
The roadmap does not see a rollout plan for the feature to use copy jobs (preview) in the data pipeline at the moment:
What's new and planned for Data Factory in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
You can give feedback on Home.
Data Factory Dataflow Gen2 also offers incremental importing. This feature allows you to load only new or updated data, thus increasing the efficiency of data loading. This is similar to the incremental import feature of Copy Job.
For more information and examples, please refer to the official documentation below:
Incremental refresh in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft Learn
If you have any other questions please feel free to contact me.
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Yang,
I really appreciate your support. I tried the OData connector in Gen2, but the authentication options were only anonymous and basic. It seems this has been updated.
I will give this a try again.
Thanks
Florian
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @fb200 ,
There is no option to use the copy job(preview) function on the data pipeline canvas.
I created a copy job(preview) in workspace and wrote its data to a warehouse, which I think is a data extraction as well as usage process, and dataflow and pipeline can be used instead.
One small note: when using multiple pipelines in one pipeline please select the “invoke pipeline(legacy)” option instead of the “invoke pipeline(preview)” option, which is currently only supported with ADF & Synapse pipelines.
Here's a screenshot of my test running in pipeline with copy job(preview) using dataflow instead.
The preview feature may have some limitations, so use an alternative if one is available.
If you have any other questions please feel free to contact me.
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
Helpful resources
Join us at the Microsoft Fabric Community Conference
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
Microsoft Fabric Community Conference 2025
Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.
Subject | Author | Posted | |
---|---|---|---|
Monday | |||
09-16-2024 05:17 PM | |||
09-27-2024 12:31 PM | |||
04-25-2024 03:59 AM | |||
10-23-2024 02:30 PM |
User | Count |
---|---|
7 | |
5 | |
2 | |
2 | |
1 |
User | Count |
---|---|
13 | |
8 | |
5 | |
5 | |
5 |