Forum Discussion
limit of 400 concurrent Copy Data activities
Hi, I’m experiencing a concurrency issue when running my orchestrator in Fabric. I’m using the Trial version with an F64 SKU. The error is as follows:
There are substantial concurrent copy activity executions which is causing failures due to throttling under subscription XXXXX-XXXXXX-XXXXX, region ne and limitation 400. Please reduce the concurrent executions.
ErrorCode: UserErrorWithLargeConcurrentRuns.
If I increase the capacity, should this issue be resolved? In other words, is capacity related to the concurrency level?
Thanks.
Based on the documentation,
https://learn.microsoft.com/en-us/fabric/data-factory/data-factory-limitations
that seems to be an issue at fabric level
2 Replies
Based on the documentation,
https://learn.microsoft.com/en-us/fabric/data-factory/data-factory-limitations
that seems to be an issue at fabric level- Hello AgustinPlain
Adding few other details in additiin to NandanHegde - Fabric allocates computing resources through CU providing a different amount of capacity
- One Capacity Unit = two Apache Spark VCores for compute-intensive operations
Capacity defines the ability of resources to perform activities or produce output, but does not directly control concurrency limits
The 400-activity concurrent execution limit exists independently of your purchased capacity SKU
Increasing your capacity from F64 to a higher SKU will provide more computational power but will not increase the 400 concurrent Copy activity limit
Potential solutiom would be :
Implement staggered pipeline scheduling to ensure that not all Copy activities attempt to run simultaneously
- Hello AgustinPlain
