Hello,
it would be great to have a option to set a parameter for how much jobs (actually table copy activities) can be started in parallel.
This would allow the user to overcome issues for example to overload on-prem datagateways with to much parallel jobs so that the gateway goes down. Currently this is not possible and Fabric "steers" the job by itself which might lead to gateway overload.
PS: yes that might not be the case for incremental loads but for first initial full loads this would come handy.
Thanks!
1 Comment
- I agree with this 100% We break one of our largest source systems down into three Copy Jobs - Full Table Copies - Standard Incrementals - Custom Incrementals The full copies are small config and lookup tables that I will happily smash 9-10 parallel runs on. The custom incrementals when doing an initial full load have billions of records. I would far rather run 1-2 at a time instead of smashing the DB with 9 parallel jobs, melting polar ice caps and contributing to global warming 😂
