Really bad throughput on copy data step copying data from SAP Hana into Lakehouse

[Bikematt]

Hi,

Our team is migrating from Synapse to Fabric.  Several of our datasets come from SAP Hana via an on prem data gateway with an ODBC connector.  We are seeing some pretty abysmal throughput in the copy data step in Fabric compared to the same step/query in Synapse.  It's worth noting that we have two different OPDGs being used for the two platforms.  We also have some datasets where we use dynamic range partitioning and some where we don't.  Are there any common pitfalls that explain the really bad copy data performance? Screenshots below:

Our Fabric throughput:

Our Synapse throughput:

[v-lgarikapat]

Hi @Bikematt ,

Thanks for reaching out to the Microsoft fabric community forum

@lbendlin 

Thanks for your prompt response

@Bikematt  in addition to @Bikematt  suggestions below, here are a few troubleshooting steps that may help you resolve the issue.

1.Check the ODBC connector and driver settings. Make sure you’re using the right, up-to-date ODBC driver for SAP HANA. Older or mismatched drivers can seriously throttle performance. Also, double-check the DSN settings like fetch size and timeout settings they matter more than you think.

2. Compare the two gateways. You mentioned Synapse and Fabric are using different on-prem data gateways. Here’s a hunch: one might be a rockstar, and the other… not so much. Look at

• CPU and memory usage on each box
• Network speed and any signs of throttling
• Gateway versions (old versions can be bottlenecks)

3. Tune the source query. If Fabric is taking forever to “wait for source,” that could mean SAP HANA is choking on the query. Run the same query directly on SAP HANA and see how fast it returns. If it’s slow there too, then maybe:

• The query needs an index or two
• The logic needs rework fewer joins, fewer wildcards, tighter WHERE clauses
• Or the table’s just massive and needs filtering or pagination

4. Revisit partitioning. Dynamic range partitioning can save you or sink you. If you’re not partitioning, you’re stuck with single thread pulls. If you are, and performance is still bad:

• Check if your partition column has good spread (no skew)
• Don’t go wild with too many partitions; the gateway will cry

5. Check staging settings in Fabric. Some copy activities in Fabric might be defaulting to stage data in intermediate storage before final delivery. That adds overhead. Poke around in the copy settings and see if staging can be toggled off for a test.
6. Test small first. Try copying a very small table or subset of rows. If even that’s crawling, it's not the data it’s the setup. This is your baseline test.
7. Network and firewall checks. Especially from the Fabric gateway to SAP HANA. Any packet loss, DNS lag, or handshake delay will compound massively over large data copies.
8. Peek at logs. Gateway logs and Fabric pipeline activity runs often contain gold nuggets warnings, slow steps, retry attempts. Don’t skip them.

For references

Copy activity performance and scalability guide - Microsoft Fabric | Microsoft Learn

Copy data from SAP HANA - Azure Data Factory & Azure Synapse | Microsoft Learn

Integrate your SAP data into Microsoft Fabric | Microsoft Fabric Blog | Microsoft Fabric

Solved: Re: Write data from Fabric Onelake to SAP on-prem/... - Microsoft Fabric Community

Best Regards,

Lakshmi Narayana

[Bikematt]

Thanks all for the responses and suggestions! I'll review these steps and follow up once we figure out the culprit.

[lbendlin]

Are these different gateways comparable from a network topology and latency perspective? Are your Synapse and Fabric location clusters comparable?

[Bikematt]

I found this previous post, but with no solution offered: Re: Data pipeline time out issue copying data from... - Microsoft Fabric Community