topic Re: Copying data gets failed in Pipelines

Copying data gets failed

Anonymous — Mon, 26 Feb 2024 08:59:23 GMT

Hi,

I was copying data from postgres to lakehouse in fabric. Some tables get easily copied but for some tables the task gets failed det to the following error:

Failure happened on 'Source' side. 'Type=Npgsql.NpgsqlException,Message=Exception while reading from stream,Source=Npgsql,''Type=System.IO.IOException,Message=Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.,Source=System,''Type=System.Net.Sockets.SocketException,Message=A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond,Source=System,'

Re: Copying data gets failed

v-cboorla-msft — Mon, 26 Feb 2024 10:55:39 GMT

Hi Anonymous,

Thanks for using Fabric Community.

The error you're encountering, "Npgsql.NpgsqlException: Exception while reading from stream," indicates that the Dataflow Gen 2 task couldn't establish a stable connection with the PostgreSQL source while copying data to the lakehouse in Fabric.

There is a default timeout of 30s for requests with a postgres connection. You can add Timeout=600;CommandTimeout=0; to your connection string .

Timeout is to establish the connection, and CommandTimeout is the timeout for the command itself (in second, 0 means infinity)

You can refer to this link for more details:

https://www.npgsql.org/doc/connection-string-parameters.html

I hope this information helps. Please do let us now if you still have any further questions.

Thanks.

Re: Copying data gets failed

Anonymous — Mon, 26 Feb 2024 11:27:10 GMT

Hi v-cboorla-msft, where should I add Timeout and CommandTimeout parameters, can you help me with this.

Thanks.

Re: Copying data gets failed

HimanshuS-msft — Mon, 26 Feb 2024 18:24:38 GMT

Hello Anonymous ,
You can make the update as shown the snapshot below .

Thanks
Himanshu

Re: Copying data gets failed

v-cboorla-msft — Mon, 04 Mar 2024 04:40:56 GMT

Hi Anonymous

Thanks.

Re: Copying data gets failed

v-cboorla-msft — Tue, 05 Mar 2024 09:44:55 GMT

Hi Anonymous

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
If you have any question relating to the current thread, please do let us know and we will try out best to help you.
In case if you have any other question on a different issue, we request you to open a new thread.

Thanks.

Re: Copying data gets failed

kesavraj — Tue, 07 May 2024 18:30:04 GMT

Linked service Edit options :