topic Performance issue of Copy Activity in fabric in Data Engineering

Performance issue of Copy Activity in fabric

imranshabbir — Fri, 12 Jul 2024 08:19:50 GMT

I'm facing performance issues in copy activity as I'm trying to fetch data from synapse to lakhouse but the thorughput I'm getting is in kbs. I have implemented the dynamic range but still facing the same issue.

Re: Performance issue of Copy Activity in fabric

Anonymous — Mon, 15 Jul 2024 02:01:32 GMT

Hi @imranshabbir,

Can you please share some more detail information about this operation and configuration? They should help us clarify your scenario and test to troubleshoot.

For example:

What type of data source are you used to copy data? Is this data source stored on cloud or on-premise? Does these operations required a gateway, and where the device that installed the gateway hosts?

Regards,

Xiaoxin Sheng

Re: Performance issue of Copy Activity in fabric

Anonymous — Thu, 25 Jul 2024 08:40:19 GMT

HI @imranshabbir,

Any update on this? Did this issue still appears on your side?
Regards,

Xiaoxin Sheng

Re: Performance issue of Copy Activity in fabric

imranshabbir — Thu, 15 Aug 2024 07:03:23 GMT

We are using synapse as source and fabric lakehouse as target. And yes these operations require a gateway.

Re: Performance issue of Copy Activity in fabric

Anonymous — Thu, 15 Aug 2024 08:35:20 GMT

HI @imranshabbir,

So you means these data are stored on azure synapse side and mapping with on-premise data source based on data gateway?
How did the network connection status between azure and gateway? Did these source and gateway managed on the same data regions?
They may cause the addition cost and affect performance when executed the connect and getting data steps.
Regards,

Xiaoxin Sheng