Forum Discussion
Issue with Upsert Table Action in Metadata-Driven Pipeline to Lakehouse Table
- 1 year ago
Hi parasol54
Warehouse tables in Fabric Data Warehouse = Lakehouse Tables.
That means Whatever you are doing in Lakehouse ( such as creating Delta table) is also applies to Data Warehouse Tables. ( Unified Data )
Lakehouse Delta Tables
Data Warehouse Tables
No limitation applies. Whether you create a Lakehouse or Warehouse Tables. I would highly recommend you to use Notebooks and Delta Tables. That way you would understand Data Lake and how it migrated to Delta Lake.
- 1 year ago
Hi parasol54 ,
Thanks for reaching out to the Microsoft fabric community forum.
lbendlin ,
Thanks for your prompt response
Since the Copy Activity in Microsoft Fabric doesn’t currently support the 'Upsert' action for Lakehouse Delta tables, a practical workaround is to load your data into a staging table using 'Overwrite' mode, then use a notebook (SQL or Python) to perform a merge operation into your destination table based on defined key columns this approach simulates upsert behavior by updating matching records and inserting new ones, and it integrates well into a metadata-driven pipeline if you extend your control table to include merge rules.
Lakehouse and Delta Tables - Microsoft Fabric | Microsoft Learn
Options to get data into the Lakehouse - Microsoft Fabric | Microsoft Learn
Best Regards,
Lakshmi Narayana
There is only overwrite mode of delta tables. It means there is no upsert needed. The reason is In Memory engine of Linux Engine aka Windows Engine.
You have to understand the whole concept of apache spark, Delta lakes and how they work.
There is no overwrite either. There's versioning and time travel and vacuuming etc.