Forum Discussion
Streaming Architecture
Hello binitafulpagare ,
Welcome to this Microsoft Fabric community forum.
Microsoft Fabric Real-Time Intelligence is the solution where you want to work with data that arrives or is updated in seconds instead of hours or even days.
Using items like the Eventstream, data from many real-time sources can be ingested. The Eventhouse is an excellent timeseries database, optimized for storing and querying events, immutable observations, facts.
Visualizations via Real-time dashboards and Maps add insights. Using the Activator and Business events, you can automatically react and take action.
Coping with low latency and high throughput is typically the challenge when working with real-time data. Because all items mentioned automatically pick up events, keep them flowing through the solution, delays are kept at a minimum. It's normal to see a lag of only a few seconds between an event being created and it appearing on the screen. This includes the time needed for connectivity between the event source (eg a sensor) and the Eventstream (as seen here).
The Fabric (RTI) resources scale up and down with the throughput need, automatically (this is the scalability part). This will mean more costs, but I expect a higher throughput means more value, too. Yes, a real-time solution needs more resources than a batch solution (because data arrives 24/7, so resources are running 24/7), so monitor and figure out whether this brings value to your use cases.
Reliability goes hand-in-hand with monitoring. Regarding real-time solutions, you are relying much more on trusting the solution that generates the events. You also rely on interfaces, the message contracts, access management, etc. So, just like in any solution, you need to test and monitor. Fabric RTI offers several solutions out of the box for this. To pick one, the Eventstream ingestion can be monitored via an activator. There is a rule available for the heartbeat. If no data arrives, eg. for more than 2 minutes, an alert is generated.
To wrap up, any market where real-time insights are needed, those solutions can benefit from Microsoft Fabric RTI. The Internet of Things is a good example, but I can also name financial systems, offshore, smart buildings, auditing, logging, shopping information, etc.
I recommend trying out Fabric RTI with a clear use case in mind. Then you will understand why Fabric RTI add value in any solution.
---
If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.
Cost
Scalability
Reliability
Hello binitafulpagare ,
Welcome to this Microsoft Fabric community forum.
Microsoft Fabric Real-Time Intelligence is the solution where you want to work with data that arrives or is updated in seconds instead of hours or even days.
Using items like the Eventstream, data from many real-time sources can be ingested. The Eventhouse is an excellent timeseries database, optimized for storing and querying events, immutable observations, facts.
Visualizations via Real-time dashboards and Maps add insights. Using the Activator and Business events, you can automatically react and take action.
Coping with low latency and high throughput is typically the challenge when working with real-time data. Because all items mentioned automatically pick up events, keep them flowing through the solution, delays are kept at a minimum. It's normal to see a lag of only a few seconds between an event being created and it appearing on the screen. This includes the time needed for connectivity between the event source (eg a sensor) and the Eventstream (as seen here).
The Fabric (RTI) resources scale up and down with the throughput need, automatically (this is the scalability part). This will mean more costs, but I expect a higher throughput means more value, too. Yes, a real-time solution needs more resources than a batch solution (because data arrives 24/7, so resources are running 24/7), so monitor and figure out whether this brings value to your use cases.
Reliability goes hand-in-hand with monitoring. Regarding real-time solutions, you are relying much more on trusting the solution that generates the events. You also rely on interfaces, the message contracts, access management, etc. So, just like in any solution, you need to test and monitor. Fabric RTI offers several solutions out of the box for this. To pick one, the Eventstream ingestion can be monitored via an activator. There is a rule available for the heartbeat. If no data arrives, eg. for more than 2 minutes, an alert is generated.
To wrap up, any market where real-time insights are needed, those solutions can benefit from Microsoft Fabric RTI. The Internet of Things is a good example, but I can also name financial systems, offshore, smart buildings, auditing, logging, shopping information, etc.
I recommend trying out Fabric RTI with a clear use case in mind. Then you will understand why Fabric RTI add value in any solution.
---
If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.
Cost
Scalability
Reliability
Hi svelde,
Thank you for the comprehensive explanation and for sharing practical insights into Microsoft Fabric Real-Time Intelligence.
I found your explanation of Eventstream, Eventhouse, Real-Time Dashboards, Activator, and Business Events particularly helpful. The point about balancing low latency, scalability, and cost is especially important when designing production-grade real-time solutions.
I have one follow-up question. As organizations expand their real-time workloads across multiple business domains, what architectural or governance practices have you found most effective for managing Eventstreams, Eventhouses, and alerting rules at scale? For example, do teams typically centralize these resources in a shared platform, or do they maintain separate environments for each business domain to simplify management and governance?
Thank you again for sharing your expertise. Your real-world insights into monitoring, scalability, and reliability are very valuable, and I'd also be interested in hearing how other community members are approaching enterprise-scale Real-Time Intelligence implementations.
