Live Stream Data in EventStreams Fabric

[uriel-kluk]

Rationale

Currently, while designing transformations and actions in EventStreams Fabric, the tool queries static or cached data. Operators can refresh manually to see new rows, but this does not reflect real-world telemetry scenarios where data may arrive irregularly and suddenly. Operators need visibility into this changing data context as it happens.

Introducing Live Stream Data into EventStreams Fabric provides a continuous data feed view, allowing operators to design and test transformations on evolving data, ensuring readiness for real-time operational use cases.

Scope

This idea introduces the ability to:

• Display a live data preview viewer inside the design environment, with raw data without applying a schema.

• Continuously update rows in real time as telemetry data arrives.

• Allow an operator to select a particular row and either:

  • Apply an existing schema to it, or

  • Ask the system to auto-infer a schema for new transformations.

This capability does not replace existing refresh and batch query functionality; it augments it with real-time interactivity for scenarios requiring immediate insight.

User Scenarios

• Scenario 1: Emergency Telemetry
  A firetruck sends regular telemetry (engine temp, fuel, GPS). During a live drill, new alarm signals appear. Operators use the live preview to detect new fields, select them, and quickly apply schema inference.

• Scenario 2: IoT Device Rollout
  During onboarding of new devices, unexpected fields arrive in telemetry (e.g., new firmware versions or error codes). Operators select rows in the live feed, trigger auto-schema inference, and extend the transformation pipeline on the fly.

• Scenario 3: Business Monitoring
  In a retail environment, live telemetry streams from devices (POS systems, sensors) are monitored. When special events occur (e.g., promotional triggers), operators test new actions against these events directly in the live feed preview.

Benefits

• Faster design of new data conditions without waiting for refresh cycles.

• Improved schema agility when unanticipated telemetry arrives.

• Debugging: Enhanced operator confidence by visually validating live events during transformation design.