Forum Discussion
Updating
Dataflows only offers two data destination operations:
- Append
- Replace
In that sense, it doesn't support any sort of delete operations or "Upserts" to update existing records.
If your destination was a SQL Database, you could perhaps leverage a combination of a Data Pipeline, a Dataflow Gen2 and a SQL Stored procedure to perhaps load the data inside of a "staging table" in your SQL Database and then run a stored procedure that will take that table and figure out, based on your own logic, what records to delete, update or add to your main table.
I'm not sure how feasible that is with a Lakehouse as a destination, but you could try asking more questions specific to Lakehouse in the Lakehouse community forum (link below):
Hi elavery95 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help .
Hi elavery95 ,
We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .
Thanks