topic Re: Failed lookups and duplicate insert in copy activity. Temporary server problem? in Pipelines

Failed lookups and duplicate insert in copy activity. Temporary server problem?

JE_test — Mon, 29 Apr 2024 07:54:35 GMT

Hi!

My pipeline have been running well for about a week, but tonight some of the tables that were supposed to be loaded failed due to this error in a lookup:

ErrorCode=SystemErrorFailedToGetAccessTokenByUserAuth,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Internal Server Error. Please retry later. If issue consist repro, please contact us.,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.DataTransfer.TokenService.Client.Models.ErrorResponseException,Message=Operation returned an invalid status code 'InternalServerError',Source=Microsoft.DataTransfer.TokenServiceClient,'

The lookup is referencing a table in my Data Warehouse. And most of the lookups worked, but some failed. I wonder what this error means? And do you have some suggestions on how to avaoid tehm in the future?

Also, after the lookup I run a copy activity. And for three of my tables (where the lookup worked) the data got inserted twice into my Lakehosue table, causing duplicates. But I have the settings on "overwrite" so that shouldn't happen:

How can this be avoided in the future?

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

v-cboorla-msft — Tue, 30 Apr 2024 07:29:33 GMT

Hi JE_test

At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.
Appreciate your patience.

Thanks.

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

v-cboorla-msft — Tue, 30 Apr 2024 17:56:31 GMT

Hi JE_test

Apologies for the inconvenience. Internal team has suggested to open a support ticket.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Thank you.

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

v-cboorla-msft — Wed, 01 May 2024 18:03:54 GMT

Hi JE_test

We haven’t heard from you on the last response and was just checking back to see if you've had a chance to submit a support ticket. If you have, a reference to the ticket number would be greatly appreciated. This will allow us to track the progress of your request and ensure you receive the most efficient support possible.

Thanks.

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

JE_test — Thu, 02 May 2024 07:58:42 GMT

I created a ticket now: 2405020050000585

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

JE_test — Mon, 06 May 2024 09:43:26 GMT

I got an answer from the support team:

Thanks for sharing the information. Firstly, apologies for the inconvenience caused. After some time investigating on the issue and checked with the backend logs, we have come to know that the issue has been caused due to outage in North Europe region during that timeframe and it is mitigated now.

Re: Failed lookups and duplicate insert in copy activity. Temporary server problem?

v-cboorla-msft — Mon, 06 May 2024 09:45:49 GMT

Hi JE_test

Glad that you query got resolved and thank you for sharing the same with the community as it can be helpful to others.
Please continue using Fabric Community for further queries.

Thank you.