Fabric CopyJob fails execution after initial run is successful

[MrHappyHead]

Hello,

We have encountered a problem with Fabric CopyJobs loading from Azure SQL database to a Fabric Lakehouse.  

The problem only seems to have started in the last 24 hours, we have made no changes to our copy jobs during that time - has there been a deployment from MS side recently?

Symptoms:

1) Pre-existing copy jobs that have been executing without issue over the last 2 weeks have failed this morning UK time

2) Creating a new CopyJob loading data from SQL Azure database to a Fabric lakehouse works successfully on first execution, but, fails on subsequent executions.

The CopyJobs have been setup as Full load and overwrite the data.

We get this error:

Operation on target CopyJobActivityLoop failed: Activity failed because an inner activity failed; Inner activity name: CopyData_final, Error: Failure happened on 'destination' side. ErrorCode=InvalidParameter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The value of the property 'DataFileListingWrapper' is invalid: 'The required property is not specified.
Parameter name: DataFileListingWrapper'.,Source=,''Type=System.ArgumentException,Message=The required property is not specified.
Parameter name: DataFileListingWrapper,Source=Microsoft.DataTransfer.ClientLibrary,'

Please help!

[MrHappyHead · Accepted solution]

We have raised a ticket as suggested.

It transpired that it was in fact a problem with our on prem Gateway - it needed a newer version of of software installing.

[v-veshwara-msft]

Thanks for sharing that the issue was identified as an outdated version of the on-premises gateway. Updating the software should help ensure smooth execution of your CopyJobs going forward.

Please feel free to reach out if you need any further assistance.

If this resolved your issue, please mark your reply as Accepted solution to help other members in the community facing a similar issue.

Thank you.

[v-veshwara-msft]

Hi @MrHappyHead ,

Thanks for using Microsoft Fabric Community,

Sorry to hear that you're experiencing issues with your Fabric CopyJobs when loading data from Azure SQL Database to a Fabric Lakehouse.

The error message related to 'DataFileListingWrapper' suggests an issue on the destination side. While this isn't a widely reported issue, similar errors have been observed in scenarios where data movement processes encounter internal service errors.

It's possible that a deployment from Microsoft's side could be impacting your Fabric CopyJobs.

I recommend reaching out to Microsoft support by raising a support ticket from here for further investigation or any updates related to recent deployments.

Let us know if you need any additional guidance.

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and a kudos would be appreciated.

Best regards,

Vinay.