Better Fabric architecture for critical real-time intelligence scenario

[pmscorca]

Hi,

I need to design a Fabric architecture for a critical real-time intelligence scenario, like f.e. an air traffic control context.

This architecture has to be as better as possible in terms of performance: the data must flow very very quickly from the source in order to produce any KPIs and insights in a report/dashboard (Power BI?), with a time within 2-3 seconds!

The input are represented by event messages with a size less 1 MB from an Azure Event hubs: it is possible to send more messages/second.

With these requirements it is crucial to choice the right architectural component, to set the related options and to apply the best practices.

Any helps to me in order to design the right real-time intelligence architecture for a critical scenario, please? Many thanks

[datacoffee · Accepted solution]

Hello

With the use of Eventstream to listen to the Event Hub messages and the possibility to scale up the throughput directly from Fabric you should be able to get a really fast solution.

Schematic setup:

Event Hub --> |Fabric | --> Eventstream --> Eventhouse -->Real-Time Dashboard or KQL queryset --> Reflex item with alert

I've done this a lot of times and the "delay" here is around 500 ms (aprox 0.5 seconds)

All in all it depends on the setup, the use case and other requirements. Also the Fabric capacity must have enoug capacity units to use for compute and message handling. I don't know the amount of data in your setup, but I would start at F64 to see if that is enough.

Let me know if above works 🙂

[lbendlin]

Our experience is that Kusto has about an 8 minute delay.

[pmscorca]

Hi,

many thanks for sharing your real experience.

If possible, could you indicate some details about the architecture applied, options and practices used, please? Thanks

[lbendlin]

Fabric Capacity Metrics App.