Handling late and out-of-order events in Eventstream windows and Eventhouse aggregations?
I'm coming from Apache Flink, where late and out-of-order events are handled with event-time watermarks and allowed lateness. I'm trying to understand the equivalent approach in Fabric Real-Time Intelligence. Scenario: IoT devices send telemetry with an event timestamp. Because of network delays, some events arrive several minutes late, and occasionally out of order. I need accurate 5-minute aggregates (average and max per device) for dashboards and Activator rules. Questions: Eventstream: when using windowed aggregations (tumbling or hopping), are windows based on event time or arrival time? Is there a setting for out-of-order tolerance or late-arrival handling, and what happens to events that arrive after a window has closed: are they dropped, or do they produce a corrected result? Eventhouse: if I aggregate in Eventhouse instead, for example with a materialized view using bin(EventTime, 5m), are late events included automatically when they arrive? Any caveats with materialized views and late data? Which layer is better for aggregations when correctness under late data matters more than lowest latency? Activator: if a rule has already fired on an aggregate that later gets corrected by late data, is there a pattern for handling that, or should rules only run on finalized windows?
Changing Ingestions from JSON
I have an EventStream receiving IoT Data out of JSONs. In my KQL table the columns timestamp, deviceId and for example x and y exist. The data from one sensor gets ingested in the columns timestamp, deviceId and y. No x since there is no data in the JSON for that. I dont work with a Mapping because it only makes it more complicated and the names in the JSON and in the KQL table are identical. No I changed the named for data from y to x when creating the JSON. Since there is no mapping and the column already exists in the KQL table the data should be saved now in column x. My problem is, it doesn't the data is not in x nor in y. It seems to be gone. Is that a bug in fabric? Can I solve this?
Anomaly Detector is disabled
Hello, I have a big Table in which we have our TimeSeries data from various devices. Since not every device sends the same data we have a big schema of numeric columns that are not filled for every deviceId. Is that the reason why there is no Anomaly Detector in my EventHouse? In the schema I can see that columns for the data are int or real. Some columns are string, but that shoudn't be the reason, right? Has anyone faced the same issue?Fabric Business Events: what delivery guarantees and replay pattern should we design for?
Hi all, I am testing the newer Business Events capability in Fabric Real-Time Intelligence and trying to understand what reliability assumptions should be made for a production design. The pattern I am looking at is roughly: Eventstream → Business Event → Activator → downstream action / User Data Function with Eventhouse enabled so the published business events are also retained for historical analysis. The current documentation explains the publisher/consumer model and shows how Eventstream can publish a governed business event that Activator then consumes. What I have not been able to find clearly documented is the delivery contract between the published business event and its consumers. A few things I am trying to clarify: If an Activator consumer or downstream action is temporarily unavailable, does Fabric retry delivery of the business event? Should consumers assume at-least-once delivery and therefore be designed to handle duplicate events, or is a different delivery model used? Is event ordering guaranteed in any scope, for example for events from the same Eventstream publisher? Since published business events can also be retained automatically in Eventhouse, is that retained history intended to support replay/reprocessing after a consumer outage, or is it primarily an analytical record and replay would need to be implemented separately? Are there documented retry or delivery-retention windows that should be considered when designing an operational workflow? I am mainly trying to understand what a resilient production pattern should look like when the business event triggers something with side effects, where processing the same event twice or silently missing an event would matter. Would you generally make the downstream consumer idempotent and treat Eventhouse as an audit/recovery store, or is there a more Fabric-native pattern for this? Interested to hear how others are approaching this with Business Events and Activator.
