Data Pipelines-Performance

There can be multiple reasons due to change in performance time for the same amount of data :

1) What was the load / active jobs or queries on the SQL server between the 2 instances?

2) Were there any other Fabric jobs being executed between the 2 instances as the capacity is being shared

Thanks for the reply from NandanHegde , please allow me to provide another insight:

Hi Anonymous ,

Based on your description, there are a number of factors that can contribute to this condition, and you can troubleshoot from these perspectives as follows:

• Check the CPU, memory, and network bandwidth utilization of the F64 SKU capacity during the pipeline run. If resource utilization is high, it may lead to performance degradation. Tools such as Azure Monitor can be used to check these metrics.
• Check network bandwidth and latency between ADLS and SQL Server. Network issues may cause data transfer to slow down. You can use network monitoring tools to diagnose network performance issues.
• Check if there are other concurrent tasks or pipelines running at the same time, causing competition for resources. Ensure that sufficient resources are available when running critical tasks.

For more details, you can read below document:

Azure Monitor overview - Azure Monitor | Microsoft Learn

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.