topic Re: Upsert operation Eventstream into a delta table in Eventstream

Upsert operation Eventstream into a delta table

Anonymous — Wed, 17 Jul 2024 04:37:21 GMT

how do i do an upsert operation in the eventstreams to a delta table? i dont see in the transformation event has the option to do an upsert using Key.
Does the only way to do upsert is using azure event hub with Spark job?

Re: Upsert operation Eventstream into a delta table

JohanLudvig — Wed, 17 Jul 2024 12:11:07 GMT

Upsert isn't supported. And not sure if that support will come in the future - but maybe if Eventstreams get to parity with Stream Analytics it will be possible.

Re: Upsert operation Eventstream into a delta table

Anonymous — Thu, 18 Jul 2024 07:19:45 GMT

thank you. so how do i do an upsert to a delta table then?

Re: Upsert operation Eventstream into a delta table

JohanLudvig — Thu, 18 Jul 2024 08:13:36 GMT

For now I would run a notebook (Spark.)

Or - and mind you, I haven't tested this on Onelake, only SQL - use a UDF in Kusto that upserts to a OneLake shortcut/external table (https://learn.microsoft.com/en-us/fabric/real-time-intelligence/onelake-shortcuts?tabs=onelake-shortcut)

Re: Upsert operation Eventstream into a delta table

joezhang-es — Mon, 22 Jul 2024 18:24:11 GMT

for event stream and lakehouse integration today, append is the already operator supported. there is one backlog item tracking the "delete" and "upsert" operator support for lakehouse