Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Did you hear? There's a new SQL AI Developer certification (DP-800). Start preparing now and be one of the first to get certified. Register now

yexu

Higher Performance with Copy job in Fabric Data Factory Auto Partitioning (Preview)

If you haven’t already, check out Arun Ulag’s hero blog “FabCon and SQLCon 2026: Unifying databases and Fabric on a single, complete platform” for a complete look at all of our FabCon and SQLCon announcements across both Fabric and our database offerings. 

Introduction

Copy job is the go-to solution in Microsoft Fabric Data Factory for simplified data movement across multiple clouds. With native support for bulk copy, incremental copy, and change data capture (CDC) replication, it can handle a wide range of movement scenarios through an intuitive, easy-to-use experience.

The most common ask from data teams is straightforward: make data movement faster. Today, we're announcing two enhancements to Copy job in Microsoft Fabric Data Factory that deliver just that:

  • Auto-partitioning for large datasets — Copy job now can automatically partitions large tables during data movement, delivering major throughput gains without any manual configuration.
  • 2X faster copy performance by default when writing to Lakehouse tables— achieved by disabling V‑Order by default during ingestion, reducing write overhead and accelerating data loads with no additional configuration required.

Auto-Partitioning: Faster large-table loads, out of the box

When you move a table with millions of rows, the difference between a single-threaded read and a partitioned parallel read can be the difference between hours and minutes. Partitioning splits a large dataset into smaller chunks that can be read and written concurrently, dramatically increasing throughput.

The challenge is that partitioning has traditionally been a manual exercise. In most data movement tools, you need to:

  1. Identify a suitable partition column — typically a numeric or date column with even distribution
  2. Define partition boundaries — specifying ranges, row counts, or hash buckets
  3. Tune parallelism — setting the number of concurrent threads or workers
  4. Test and iterate — verifying that your partition strategy doesn't create hotspots, skewed workloads, or source-side pressure
This is engineering effort that scales with the number of tables you're moving. If you have 50 large tables across five source systems, you're tuning 50 partition configurations — and re-tuning them whenever data volumes or distributions change.

Copy job now handles all of this automatically. When Copy job detects a large dataset, it intelligently analyzes the source schema and data characteristics to determine the optimal partitioning strategy — selecting the right partition column, computing balanced boundaries, and executing parallel reads — all without any user input.

What this means for you:

  • No partition configuration — You don't specify columns, ranges, or parallelism. Copy job analyzes the source and makes the optimal decision.
  • Adaptive throughput — The partitioning strategy scales with the data. Larger tables get more partitions; smaller tables proceed without partitioning overhead.
  • Consistent performance across tables — Whether you're copying a 100-row lookup table or a 500-million-row transaction log, Copy job applies the right strategy automatically.
Supported connectors for auto-partitioning: Amazon RDS for SQL Server, Azure SQL Database, Azure Synapse Analytics (SQL Pool), Fabric Data Warehouse, SQL database in Fabric, SQL Server, and Azure SQL Managed Instance.

Supported copy mode for auto-partitioning: Watermark-based incremental copy including both initial full copy and incremental copy.

Coming from Azure Data Factory?

If you're familiar with ADF's Copy activity, you already know the tradeoff: you can achieve great performance, but it typically takes hands-on tuning and careful run behavior
Capability Azure Data Factory (Copy Activity) Copy Job in Fabric
Partitioning Manual. You choose the partition option, partition column, lower/upper bounds, and partition count in the Copy activity settings. Supported for specific source types (for example: SQL, Oracle, Netezza, Teradata, SAP). Automatic. Copy job detects large datasets and applies an optimal partitioning strategy with no configuration required.
Table 1: ADF and FDF comparation for partition.

The shift: ADF gives you the tools to build high-performance copy, but you have to tune it yourself. Copy job makes high-performance copy the default.

How to get the benefit from features

Getting the benefits of these two enhancements is straightforward:

Auto-partitioning — Turn on the Auto-partitioning toggle under Advanced settings in your Copy job.

Snapshot_of_enabling_auto_partitioningSnapshot_of_enabling_auto_partitioning

Figure: Snapshot of enabling auto partitioning.

2X faster copy performance by default when writing to Lakehouse tables — No action required. There is no code change and no configuration needed. If you still want to enable V‑Order for writes to Fabric Lakehouse tables, you can do so from the Advanced settings page in the Copy job.

Learn More

Submit feedback on Fabric Ideas and join the conversation in the Fabric Community.

Questions or feedback? Leave your thoughts in the comment section.