Pipeline Copy Activity Error

[coverturfLSH]

Hello, 

I'm getting the following error trying to run a copy activity from a lakehouse to a warehouse. The lakehouse and warehouse tables were autocreated from a dataflow. The final published dataflow is to the lakehouse. I then want to run a copy of the lakehouse data into my warehouse. However, I get the following error but can't see anything in the error message that would help me troubleshoot.

ErrorCode=DWCopyCommandOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message='DataWarehouse' Copy Command operation failed with error ''
Statement ID: {9E77DEC9-BF98-4DAD-98B6-AA5D89CE2914} | Query hash: 0xBDA3704A82B07146 | Distributed request ID: {4EDDB709-B765-4441-AC88-216817FD269A}'.,Source=Microsoft.DataTransfer.Connectors.MSSQLImport,''Type=Microsoft.Data.SqlClient.SqlException,Message=
Statement ID: {9E77DEC9-BF98-4DAD-98B6-AA5D89CE2914} | Query hash: 0xBDA3704A82B07146 | Distributed request ID: {4EDDB709-B765-4441-AC88-216817FD269A},Source=Framework Microsoft SqlClient Data Provider,' 

It had been working fine until this week when this error message started happening. 

Any thoughts?

Thanks,

Cameron

[coverturfLSH · Accepted solution]

Issue seems to be related to a lakehouse varchar field that is at the max 8000 characters. The warehouse destination is also configured for 8000 characters but something in the copy process must not be liking that many characters. 

To fix I'm unfortunately having to exclude that field from the copy, but the rest of the fields copy fine.

[coverturfLSH]

Hi Nono,

Thanks for the suggestions but none of those apply to my issue. I will try deleting the whole dataflow and starting again. 

[Anonymous]

Hi @coverturfLSH 

Error codes usually point to configuration problems.

Make sure that the copied data does not exceed the size limit of the target column.

If the number of concurrent operations exceeds the limit, the operation may fail.

Ensure that the data warehouse has sufficient resources. The operation may fail due to insufficient resources.

Token issues can also cause this error. Ensure that the token used for authentication has not expired.

If your data transfer involves network resources, make sure the necessary network and firewall Settings are correctly configured to allow the data transfer.

Sometimes, this may be a temporary problem, please try again.

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.