topic Re: Data pipeline time out issue copying data from on-premises in Pipelines

Data pipeline time out issue copying data from on-premises

govindarajan_d — Wed, 03 Apr 2024 12:36:04 GMT

Hi All,

We have a SAP HANA system in our on-premises and we have installed the On-premises data gateway to copy data into Lakehouse.

The OPDG is present in VM which has 8 GB RAM.

When we tried to extract data from SAP HANA into Lakehouse as delta table, it runs for 30-40 mins and timeouts. I even tried just 100 records but fails.

But the same thing works when I use parquet format or CSV. I can even extract 20K records under a minute in those formats. I also tried ADF with Self-hosted Integration runtime, but still the same result.

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Thu, 04 Apr 2024 10:58:38 GMT

Hi govindarajan_d
Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this. We will update you once we hear back from them.
Thanks

Re: Data pipeline time out issue copying data from on-premises

cieplag — Fri, 05 Apr 2024 10:19:05 GMT

Had the same problem, but I wasn't able to load data from on-premises SQL Server to lakehouse at all (table and files). I am using March version of on-prem data gateway running on Hyper-V virtual machine and I had to disable processor compatibility mode on it additionally for this to work.

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Mon, 08 Apr 2024 07:45:28 GMT

Hi govindarajan_d
The internal team has replied as follows: Insufficient information for troubleshooting. Customer Failed to test Connection or copy timeout? Since ADF with SHIR also encountered this timeout, it's highly likely to be due to incorrect user configuration.

Copy data from SAP HANA - Azure Data Factory & Azure Synapse | Microsoft Learn

Can you please confirm if its Failed to test connection or Copy timeout?
Than

Re: Data pipeline time out issue copying data from on-premises

marcoG — Wed, 10 Apr 2024 10:31:47 GMT

Is it the same problem?

https://community.fabric.microsoft.com/t5/Data-Pipelines/The-copy-from-sql-server-on-premise-has-been-released-but-it/m-p/3788009#M2227

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Wed, 10 Apr 2024 17:13:31 GMT

Hi govindarajan_d
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Thu, 11 Apr 2024 17:24:08 GMT

Hi @govindarajan_d
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: Data pipeline time out issue copying data from on-premises

marcoG — Thu, 11 Apr 2024 18:00:17 GMT

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Thu, 11 Apr 2024 17:58:18 GMT

Hi marcoG
I couldn't get you. Can you explain further?
Thanks

Re: Data pipeline time out issue copying data from on-premises

marcoG — Thu, 11 Apr 2024 18:00:40 GMT

Sorry i put wrong response

Re: Data pipeline time out issue copying data from on-premises

Anonymous — Thu, 11 Apr 2024 18:01:53 GMT

marcoG
No problem. Have a good day!