How to save incompatible rows to lakehouse/warehouse tables during a pipeline run

[Srisakthi]

Hello All,

I have a pipeline which has copy activity to copy data from multiple source tables to lakehouse tables. I want to copy only proper records to my lakehouse table and all incompatible rows to be saved in another lakehouse tables. I could find an option in pipeline settings to skip incompatible rows and logging it. But currently it shows only Azure Blob and ADLS. Suggest if there is any way to store those incompatible rows to lakehouse tables for further analysis? Please refer SS

Regards,

Srisakthi

[MonicaCVL]

Hi @Srisakthi,

Since this feature isn’t supported yet, you can alternatively add another activity to push data from ADLS to a lakehouse. This can be done incrementally by using watermarks and updating them.

with regards

Monica CVL

[v-kongfanf-msft · Accepted solution]

Hi @Srisakthi ,

As you said, currently it only shows Azure Blob and ADLS however, for the time being there is no better way to store incompatible rows in the Lakehouse table for further analysis.

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.