Forum Discussion
Upsert data to Lakehouse Table using copy data
Hi Alaahady ,
You need to transfer table from on-premises sql server to datalake using incremental update (upsert) . I have provided all the possible work arounds. But you are expecting Low code / No code. You can try the "Copy job" in Data Factory, makes it easy to move data from your source to your destination without creating a pipeline.
Please refer below links.
How to create a Copy job in Data Factory - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
Hello Alaahady,
here is an example of how you could do this with Dataflow Gen2. However, it is currently only partially available in preview.
That might help.
https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-incremental-refresh
Another option would be to load the data into the lakehouse and then perform an update using a notebook.
However, this naturally requires a little extra effort.
Best regards
Thank you Spaceman, I may call you suprman for promot repond.
I tried this option but I got the following message:
Incremental refresh
⚠️ It is not supported to setup incremental refresh on a query when:
it uses a default destination
or it uses a data destination that is not supported
Please update the data destination to a supported data destination (Fabric Warehouse, Fabric SQL, and Azure SQL) before setting up incremental refresh.