Forum Discussion
Fabric Copy Data activity Capacity Usage
Hi everyone,
Curerntly, I'm looking into the performance of the pipeline in Fabric and checking the CU via Capacity Metrics App. Why the DataMovement for job with duration is 44 is 1080, but for others is only 360. I thought if the duration is less than 60s, it still takes 360 only. Is the data size one of the factor, since the jobs with 44s is moving data from a SQL server, and others are files. That'd be much help if anyone could help me understand the mechanism. Thanks!!!
Hi harrybao0901,
Thanks for reaching out to the Microsoft Fabric Community forum. and thanks to sannavajjala for sharing valuable insights.
According to the official Fabric documentation, Data Movement consumption for a Copy activity is based on:
- Copy activity run duration
- The intelligent optimization throughput resources used during the Copy activity execution
The Copy activity also handles the copy process by connecting to the source, processing the data (such as serialization/deserialization, compression/decompression, column mapping, and data type conversions), writing the processed data to the destination, and providing monitoring with detailed logs and metrics.
To compare the SQL Server → Snowflake and Files → Snowflake executions, you can review the Copy activity monitoring details, including metrics such as copyDuration, dataRead, dataWritten, throughput, usedDataIntegrationUnits, and usedParallelCopies.
For more information, please refer to the below official documentation:
Pricing for Pipelines - Microsoft Fabric | Microsoft Learn
How to copy data using copy activity - Microsoft Fabric | Microsoft Learn
Monitor Copy activity - Microsoft Fabric | Microsoft Learn
I hope this helps. Please feel free to reach out if you have any further questions.
Thank you.Hi harrybao0901,
Thanks for sharing your findings. According to the Pricing for Pipelines documentation, Copy activity capacity consumption is based on both the Copy activity run duration and the intelligent optimization throughput resources used during execution, rather than duration alone.
To further analyze the differences between your runs, you can compare the Copy activity monitoring metrics such as copyDuration, throughput, usedDataIntegrationUnits, and usedParallelCopies, as these provide additional insight into the Copy activity execution.
For more details, please refer to the below official documentation:
- Pricing for Pipelines: Pricing for Pipelines - Microsoft Fabric | Microsoft Learn
- Monitor Copy activity: Monitor Copy activity - Microsoft Fabric | Microsoft Learn
I hope this helps. Please feel free to reach out if you have any further questions.
Thank you.
6 Replies
Hi,
What you're seeing is expected behavior in many Fabric pipeline scenarios. While duration is one factor in CU consumption, it is not the only factor. The Capacity Metrics App calculates DataMovement CU usage based on the underlying compute resources and data movement activities involved, not just the execution time.
In your example, the 44-second DataMovement operation consuming 1080 CU likely required more intensive data movement resources than the 11–12 second operations consuming 360 CU. The source and destination types can make a significant difference. Moving data from SQL Server often involves connectors, query execution, network transfer, data serialization, and potential transformation overhead, whereas file-based operations may require fewer resources even if the runtime appears similar.
Other factors that can influence DataMovement CU consumption include:
- Data volume being transferred
- Number of rows and columns processed
- Connector type (SQL Server, SAP, APIs, files, etc.)
- Parallelism used by the pipeline
- Network and gateway involvement
- Data transformation or staging operations performed during movement
The common assumption that anything under 60 seconds consumes the same amount of capacity isn't always accurate. Fabric measures resource consumption based on workload characteristics and compute utilization, not strictly on elapsed time.
To validate whether data size is the main driver in your case, compare the amount of data moved, row counts, and source types for the 1080 CU execution versus the 360 CU executions. If the SQL Server activity transferred substantially more data or required more processing, the higher CU consumption would be expected even with a runtime under 60 seconds.
If you can share the activity type (Copy Data, Dataflow Gen2, Notebook, etc.) and approximate data volumes involved, the community may be able to provide a more precise explanation of the CU calculation for that specific workload.
Thanks,
Manoj Annavajjala
it's copy data activity. i copy around 30 tables from sql server to snowflake. the table size are not the same, some only 1K, 4K rows, some could reach 10K rows or more. but all CU are equivalent 1080.
And the case above (in the topic) is betwen files to snowflake and sql server to snowflake.
Hi harrybao0901,
Thanks for reaching out to the Microsoft Fabric Community forum. and thanks to sannavajjala for sharing valuable insights.
According to the official Fabric documentation, Data Movement consumption for a Copy activity is based on:
- Copy activity run duration
- The intelligent optimization throughput resources used during the Copy activity execution
The Copy activity also handles the copy process by connecting to the source, processing the data (such as serialization/deserialization, compression/decompression, column mapping, and data type conversions), writing the processed data to the destination, and providing monitoring with detailed logs and metrics.
To compare the SQL Server → Snowflake and Files → Snowflake executions, you can review the Copy activity monitoring details, including metrics such as copyDuration, dataRead, dataWritten, throughput, usedDataIntegrationUnits, and usedParallelCopies.
For more information, please refer to the below official documentation:
Pricing for Pipelines - Microsoft Fabric | Microsoft Learn
How to copy data using copy activity - Microsoft Fabric | Microsoft Learn
Monitor Copy activity - Microsoft Fabric | Microsoft Learn
I hope this helps. Please feel free to reach out if you have any further questions.
Thank you.
