topic Eventstream flow of data in Activator

Eventstream flow of data

DennesTorres — Wed, 06 Mar 2024 13:23:21 GMT

Hi,

Is there an easy way to (1) check the flow of data on an eventstream, identifying, for example, there was an interruption of 10 minutes?

and (2) create an alert if this kind of interruption happens ?

I imagine I can do this with data activator, but I was looking for something easier.

Kind Regards,

Dennes

Re: Eventstream flow of data

Anonymous — Thu, 07 Mar 2024 09:50:32 GMT

Hi DennesTorres ,

1. Monitoring Eventstream Data Flow

To efficiently monitor the flow of data in an eventstream and identify interruptions, you can leverage the Data insights and Runtime logs features in Microsoft Fabric. These features provide a comprehensive view of the status and performance of your eventstream, including metrics like IncomingMessages, OutgoingMessages, and detailed logs for troubleshooting.

Data insights: This will help you monitor metrics such as IncomingMessages and OutgoingMessages. You can access these by selecting your eventstream node in the main editor canvas and navigating to the Data insights tab in the lower pane.
Runtime logs: For identifying specific interruptions and their duration, the Runtime logs can be invaluable. You can view these logs by selecting the relevant node and opening the Runtime logs tab. Filtering by severity or keyword might help you pinpoint the exact time and duration of interruptions.

For more detailed guidance, please refer to the Monitoring status and performance of an Eventstream item - Microsoft Fabric | Microsoft Learn

2. Creating Alerts for Interruptions

Currently, setting up direct alerts for eventstream interruptions requires a more manual approach, as specific alerting functionalities for eventstream interruptions are not directly mentioned in the available documentation.

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly.

Re: Eventstream flow of data

DennesTorres — Fri, 08 Mar 2024 17:42:45 GMT

Hi,

This is a first step, but it doesn't help too much. According to the link, it seems the data insights are more detailed for some sources. I'm using a custom app source, in this way it didn't help much, the runtime logs don't even appear for me.

The Data Insights appear for me, but I have a complain from the client about a 10 seconds interruption two days ago. To visualize this I need to put the visual to 1 week data, so the 10 seconds is not identifiable.

Is there a way to extract this data insights information so I could manipulate it in the way I need?

Kind Regards,

Dennes