thomas_pagel
2 years agoNew Member
Status:
Needs Votes
Event Processing: Derive value from complex JSON
Please allow to derive a value out of a complex JSON which is stored in a single field in your stream
Anonymous
1 year agoNot applicable
Voting on the title (and not the idea on creating a no code experience for upsert policies which came up mid thread). In my experience, Eventstreams definitely needs a way to handle more complex stuctures. It's a very typical occurrence. I would be fine with the ability to execute SQL/KQL or prefereably Typescript to do this --> Extend the "transform events" list with a scripting element. For example, we quite often quite big and complex JSONs as time series data from IoT Edge, containing shopfloor data which we need to visualize and act on in a near real-time fashion. The current processing capabilities fall short here and/or are very cumbersome to process such big structures. Using upsert policies is not ideal because it does not allow for any downstream processing of the "cleaned" JSON. For example, we would want the processed JSON to be stored in a KQL database but also have an Activator element analyze and and route it to another destination.
Recent ideas
Allow External OneLake Data Shares to Be Accepted into Multiple Fabric Item Types
Fabric External Data Sharing provides an excellent zero-copy mechanism for sharing OneLake data across tenants. However, the receiving tenant currently has to accept the share into a Lakehouse. I w...Mourak19 hours agoNew MemberNew5Views0likes0CommentsAllow OneLake Shortcut Authentication Mode to Be Changed In Place
OneLake shortcuts support passthrough and delegated authentication, but changing the authentication model currently requires deleting and recreating the shortcut. I would like shortcut authenticatio...Mourak19 hours agoNew MemberNew7Views0likes0CommentsTenant-Wide OneLake Shortcut Lineage and Impact Analysis
This is grounded in a documented limitation: shortcut lineage is currently workspace-scoped, and Microsoft specifically says lineage for shortcuts to warehouses and semantic models isn't currently av...Mourak19 hours agoNew MemberNew15Views0likes1CommentAdd an End-to-End Production Reference Architecture for Microsoft Fabric Real-Time Intelligence
I would like to see a complete end-to-end Microsoft Fabric Real-Time Intelligence reference architecture that demonstrates how the individual RTI capabilities work together in a production-oriented s...Mourak19 hours agoNew MemberNew5Views0likes0CommentsImprove CI/CD consistency for Microsoft Fabric Real-Time Intelligence Eventstreams
Microsoft Fabric Real-Time Intelligence has made strong progress with Git integration and deployment pipelines, but lifecycle management can still be inconsistent across Eventstream sources, destinat...Mourak19 hours agoNew MemberNew3Views0likes0Comments