Forum Discussion
Copy Data activity to load a SQL Server view into my Fabric Lakehouse.
It is really hard to identify it from here remotely, but there is a way for you to do some checks:
Run the view directly in SQL ServerIf the query time has increased, the source is the problem. If the query runs normally, investigate the gateway or destination.
Compare the Copy Data run diagnostics
Check:
Data read throughput
Data write throughput
Duration of internal execution stages
This identifies whether the slowdown is coming from the source, transfer, or sink.
Check the gateway or SHIR machine
Look at CPU, RAM, and network rate during execution. Any resource pressure can slow down the transfer.
Run a small test copy
Try copying:
A small subset of rows
A direct table instead of a view
If the small test is slow, the destination is likely the issue. If the small test is fast, the view or data volume has changed.
Adjust Copy Behavior
Experiment with:
Parallelism settings
Data Integration Units
Enabling or disabling staging
So in general, to troubleshoot performance problems, focus on the parts of data transfer activity. Is it due to view performance?, Due to table/index performance which that view uses? Is the SQL Server where this view sits, having overloads? Is there something on the network? Is that related with the integration runtime machine that you are using? Is that the configuration that is being used in copy activity or is that something related with the destination (in this case fabric) => Like you have to run Optimize and vacuum on delta files etc. etc. Check each of these points and do tests and try to encapsulate the problem as much as possible.
Hope that helps
Onur
😊 If this post helped you, feel free to give it some Kudos! 👍
✅ And if it answered your question, please mark it as the accepted solution.
It is really hard to identify it from here remotely, but there is a way for you to do some checks:
Run the view directly in SQL Server
If the query time has increased, the source is the problem. If the query runs normally, investigate the gateway or destination.
Compare the Copy Data run diagnostics
Check:
Data read throughput
Data write throughput
Duration of internal execution stages
This identifies whether the slowdown is coming from the source, transfer, or sink.
Check the gateway or SHIR machine
Look at CPU, RAM, and network rate during execution. Any resource pressure can slow down the transfer.
Run a small test copy
Try copying:
A small subset of rows
A direct table instead of a view
If the small test is slow, the destination is likely the issue. If the small test is fast, the view or data volume has changed.
Adjust Copy Behavior
Experiment with:
Parallelism settings
Data Integration Units
Enabling or disabling staging
So in general, to troubleshoot performance problems, focus on the parts of data transfer activity. Is it due to view performance?, Due to table/index performance which that view uses? Is the SQL Server where this view sits, having overloads? Is there something on the network? Is that related with the integration runtime machine that you are using? Is that the configuration that is being used in copy activity or is that something related with the destination (in this case fabric) => Like you have to run Optimize and vacuum on delta files etc. etc. Check each of these points and do tests and try to encapsulate the problem as much as possible.
Hope that helps
Onur
😊 If this post helped you, feel free to give it some Kudos! 👍
✅ And if it answered your question, please mark it as the accepted solution.