Microsoft Fabric - Data Factory - Update/Upsert option not available in Copy Activity for Warehouse

[Sathish133]

Hi Team,

I am trying to load data into Fabric - Synapse Data Warehouse. But I surprised there is no option in the Data Pipeline Copy activity either to Update or Upsert which normally available in Azure Data Factory for Azure SQL DBs.

Question is How are we performing the Update or Upsert option in Fabric Synapse Warehouse using Fabric Data Factory.

Azure Data Factory (Sink : Azure SQL)

Microsoft Fabric Data Factory - Data Pipeline (Copy Activity - Sink : Fabric Synapse Data Warehouse)

Regards,

Sathish Kumar Mathivanan

[jwinchell40]

@Sathish133 - I agree it would be nice if the upsert options was just baked into the Pipeline as an option.  The way that I've implemented this in our environments is just writing the data to a new delta table and passing the name of the new table as a parameter into a Notebook.  Then I just run a simple Upsert using the Merge command and it works fine for now.

Happy to share a code snippet we are using if it helps at all.

[Anonymous]

Please check below thread

Solved: upsert option missing in copy data activity - Microsoft Fabric Community

[Sathish133]

Hi @Anonymous,

Thanks for the reference link.

As per the microsoft plan, UPSERT will be available Q1 2024 in Dataflow Gen2. But no information on the data pipeline.

Regards,

Sathish Kumar Mathivanan

[mkjit256]

Hello, 
Do we have any update about that? Still DataFlow Gen 2 doesn't have the upsert option