Author: Ye Xu, Principal Program Manager
Copy job is the go-to solution in Microsoft Fabric Data Factory for simplified data movement across multiple clouds and tenants. With native support for bulk copy, incremental copy, and change data capture (CDC) replication, it can handle a wide range of movement scenarios through an intuitive, easy-to-use experience.
While Copy job makes data movement simple, it also provides the flexibility to precisely control how your data is moved. In this blog, we highlight several recent improvements that enhance this flexibility.
Expanded auto-partitioning to Oracle, SAP HANA and Lakehouse table (Preview)
We recently introduced Auto Partitioning (Preview) in Copy job to deliver significantly higher performance by automatically parallelizing data movement based on source characteristics—without requiring manual tuning.
Auto Partitioning now supports these additional connectors:
- Oracle
- SAP HANA
- Fabric Lakehouse tables
With this expansion, Copy job can automatically scale out read and write operations across these data stores, maximizing throughput with simplified experience. You no longer need to predefine partition columns or manage custom sharding logic—Copy job intelligently determines the optimal partitioning strategy for you.
- To learn more, refer to the What is Copy job in Data Factory documentation.
Edit Copy job via JSON Payloads for Maximum Flexibility (Generally Available)
We’re introducing the ability to edit Copy jobs directly through JSON payloads, giving advanced users maximum flexibility and precision when configuring your Copy job for data movement. This capability provides full flexibility in Copy job authoring, beyond what is available through the visual authoring experience.
With JSON-based editing, you can programmatically define and update Copy job configurations, making it easy to automate job edit and apply bulk changes. This is especially powerful for large-scale edit or scenarios where consistency and repeatability across environments are critical.
- To learn more, refer to the What is Copy job in Data Factory documentation.
Switch between full and incremental copy mode (Generally Available)
You can now seamlessly switch between full (batch) copy and incremental copy after a Copy job has been created.
This allows you to adapt your data movement strategy over time without recreating the job:
- Switch from incremental to full copy if the incremental column or change data capture is no longer available.
- Switch from full to incremental copy once a change tracking mechanism is identified.
Previously, this required recreating the Copy job. Now, you can make the switch with a single action, reducing friction and enabling faster iteration.
Please note, incremental copy mode always starts with an initial full copy, followed by incremental copies in subsequent runs.
- To learn more, refer to the Incremental copy in Copy job documentation.
Conclusion
Copy job keeps getting better and more powerful where it matters. With smarter performance through auto-partitioning, deeper control via JSON, and the flexibility to switch between full and incremental modes, you can move data the way your scenario demands, and without extra overhead.
If you haven’t tried these updates yet, now’s a great time to explore what Copy job can do for your pipelines. Start experimenting and see how much simpler (and faster) your data movement can be.