Data Pipeline Failing intermittently with "table doesn't exist error"

WNCPBI — Tue, 15 Oct 2024 09:26:45 GMT

Quite a simple scenario. I have a data pipeline with a lookup activity getting the list of tables from a lakehouse in the same workspace. Then a for each activity loops through these and copies them to a Data Warehouse in the same Workspace.

There are circa 1300 tables and all but about 10 - 15 copy over. I receive an error on the ones that fail of:

ErrorCode=DeltaSnapshotError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Table doesn't exist.,Source=Microsoft.DataTransfer.ClientLibrary,'

The tables absolutely exist as the names all come from the source lakehouse. I'm assuming it's a Fabric Service issue/bug. Has anyone else encountered this error? Interestingly we do a similar copy lakehouse to lakehouse without any issues so it does seem to be releated only to the Data warehouse being the destination.

Re: Data Pipeline Failing intermittently with "table doesn't exist error"

lbendlin — Tue, 15 Oct 2024 12:51:13 GMT

topic Data Pipeline Failing intermittently with "table doesn't exist error" in Pipelines

Data Pipeline Failing intermittently with "table doesn't exist error"

Re: Data Pipeline Failing intermittently with "table doesn't exist error"