Forum Discussion
Upsert data to Lakehouse Table using copy data
Hi Alaahady ,
You need to transfer table from on-premises sql server to datalake using incremental update (upsert) . I have provided all the possible work arounds. But you are expecting Low code / No code. You can try the "Copy job" in Data Factory, makes it easy to move data from your source to your destination without creating a pipeline.
Please refer below links.
How to create a Copy job in Data Factory - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
Ok.
Another option would be to use the copy activity. However, this is also in Preview.
I tested this option some time ago and it worked perfectly.
Perhaps you can also check this in your case.
However, if you have already done this, then your configuration of the copy activity would be helpful.
Here is a screenshot.
I tried to copy and upsert the data using the Data pipeline. But I got the below Error.
Hi SivaReddy24680 ,
Thank you for reaching out to the Microsoft Community Forum.
The issue you are facing is due to a configuration limitation in your current setup. You are trying to use the "upsert" operation on a Lakehouse Table via the Data pipeline. May be your current version of ODPG (OneLake Data Pipeline Gateway) does not support "upsert" for Lakehouse Tables.
Solution: Upgrade to the latest version of ODPG to enable upsert functionality. After upgrading, retry the pipeline execution with the upsert action.
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
