Copy data failing for dataverse table

[RuedigerThiel]

Hello, I'm try to setup a Copy data activity. I want copy the dirpartytable, but even the preview fails with an error 21446 and the message "Invalid table! Parquet column is not defined in delta metadata. Column name: _change_type."

Any advice how to solve the problem?

Thanks Rüdiger

[RuedigerThiel]

Unfortunately I haven't got any further update from Microsoft. 

[Anonymous]

Hi @RuedigerThiel 

Thank you very much FabianSchut for your prompt reply.

Please let us know if your issue is resolved. I look forward to you sharing the information you get!

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[RuedigerThiel]

Hello everybody, here a short summary for the issue.

I have created a ticket TrackingID#2411061420002831 with the Microsoft Support. Currently the change data capture (CDC) scenario is not supported by Fabric Copy. As a workaround the usage of the Lakehouse SQL endpoint in an SQL connector instead. I requested to be informed, when the Fabric Copy supports CDC tables.

[Vishnu1995]

Hi @RuedigerThiel , How did you resolve this?

[ajarora]

We are bringing support to do CDC based ingestion via CopyJob artifact. This is in Private Preview, and we will go live soon. Please DM me for any more details.

[vivien57]

Hello @ajarora 

Do you have any news, because the problem with the CDC seems to be still present on the copy job, I've just done the test.

Thank you very much in advance,

Have a nice day,

Vivien

[RuedigerThiel]

@ajarora Can you enable us for the private preview? Pls adv  Brgds Rüdiger

[vivien57]

Hello @ajarora 

I would like to have the details please. So it won't be compatible via a copy data in a Data Pipeline?

Thank you in advance for your feedback

Vivien

[vivien57]

Hello,

Do you have any news about solving the problem? Because when I use EndPoint SQL I have this problem

https://community.fabric.microsoft.com/t5/Data-Pipeline/Connection-to-the-SQL-EndPoint-of-the-Datave...

Thank in advance for your return,

Have a nice day,

Vivien

[RuedigerThiel]

Hello Vivien, I have no idea what might be the problem there. I assume you contact the microsoft support. 

Brgds Rüdiger

[vivien57]

Thank you for your feedback.

But have you had any feedback from Microsoft on fixing the problem for tables with the CDC?

[FabianSchut]

Hi, this topic looks similar: https://community.fabric.microsoft.com/t5/Data-Engineering/Need-to-copy-tables-from-lakehouse-to-war.... They got it working with data flow gen2 instead of copy data in a data pipeline or use the SQL-endpoint. What is the source you are looking at?

However, they also suggest to create a support ticket, since it is not a long-term solution.

[RuedigerThiel]

Ticket created, contact established, will keep you updated

[vivien57]

Hello,

Have you had any news from MS support ? Is there a solution ?

Thank you in advance for your feedback

Have a nice day,

Vivien