microsoft fabric dataflow - delta

[arkiboys2]

Hello,

in the current ADF dataflow, I use a delta sink with alter row to upsert if or delete if or insert if

is there a similar functionality in fabric dataflow?

thank you

[jigneshpatel]

1) I have marked this topic as new as it was updated last a year back. I want to know if there is any soluton for UPSERT
2) also now dataflow has increamental refresh option. Is that an alternative of upsert? 
3) When we use append with additional implementation to have only new or updated rows from source. The updated rows gets appended as new records in the destination. I want it to be updated not inserted? In this thread solution says to have extra logic in python spark script to handle this issue. Can you give more detail on that? 
4) Can we add any script in the dataflow which will search record in destination and update if found or insert if not found?


Jignesh

[arkiboys2]

thank you

[FabianSchut · Accepted solution]

Hi, currently there is no such functionality in DataFlow Gen2. You could use an append to write data incrementally to the destination and write your own Python or SQL-script to handle changes in the data.