topic Error while updating change data feed enabled table using pipeline in fabric in Data Engineering

Error while updating change data feed enabled table using pipeline in fabric

Anonymous — Fri, 13 Dec 2024 18:28:51 GMT

I have got the following error while updating a change data feed enabled table in fabric using pipeline.When I tried to update this table using note book,It worked fine but with pipeline I got the following error.I even tried configuring the minWriterVersion as a potential fix, but the issue persists.What could be the reason?

Re: Error while updating change data feed enabled table using pipeline in fabric

AndyDDC — Fri, 13 Dec 2024 20:33:42 GMT

Hi @Anonymous at any point did you upgrade the table version to a higher level than the default? Do you see any table metadata sync issues with the lakehouse sql endpoint?

Can you checl the interop guide here for Delta and the various Fabric services/features Delta Lake table format interoperability - Microsoft Fabric | Microsoft Learn

Re: Error while updating change data feed enabled table using pipeline in fabric

Anonymous — Mon, 16 Dec 2024 08:50:21 GMT

Hi @Anonymous

It sounds like you're dealing with an issue related to Delta Lake table protocol versions. Unfortunately, once a Delta table's protocol version has been upgraded, it cannot be downgraded. This is an irreversible operation.

What is the data source? Can you first check its current writer and reader version?

%%sql DESCRIBE DETAIL yourTableName;

In addition, how did you update the table using notebook? Can you share the code snippet?

Best Regards,
Jing
Community Support Team