thomas_pagel
2 years agoNew Member
Status:
Needs Votes
Event Processing: Derive value from complex JSON
Please allow to derive a value out of a complex JSON which is stored in a single field in your stream
Anonymous
1 year agoNot applicable
Voting on the title (and not the idea on creating a no code experience for upsert policies which came up mid thread). In my experience, Eventstreams definitely needs a way to handle more complex stuctures. It's a very typical occurrence. I would be fine with the ability to execute SQL/KQL or prefereably Typescript to do this --> Extend the "transform events" list with a scripting element. For example, we quite often quite big and complex JSONs as time series data from IoT Edge, containing shopfloor data which we need to visualize and act on in a near real-time fashion. The current processing capabilities fall short here and/or are very cumbersome to process such big structures. Using upsert policies is not ideal because it does not allow for any downstream processing of the "cleaned" JSON. For example, we would want the processed JSON to be stored in a KQL database but also have an Activator element analyze and and route it to another destination.
Recent ideas
OneLake Catalog full-text and content search
Why is it not possible to search OneLake by item description or even content? So I was thinking about how a user can find a table if they don't know in which lakehouse it is located? Why can't you se...TePe11 hours agoHelper IVPlanned1.1KViews21likes5Comments