Forum Discussion
Super UserEventstreams Dropping Messages When Throughput Spikes Above 15k msg/min — How to Prevent Data Loss?
We’re streaming IoT sensor data (factories) into Eventstreams.
When throughput spikes above ~15k msg/min, we start to see message drops before they reach the KQL database.
Replay diagnostics show ingestion gaps, but CPU/memory seems fine.
Is there any recommended pattern to buffer or auto-scale ingestion in Fabric RTI?
Or is this a known limitation with high-throughput streams?
Hello, please start by checking the eventstream throughput settings on the Eventstream. Do you have it set appropriately?
Configure settings for a Fabric eventstream - Microsoft Fabric | Microsoft Learn
Hello SavioFerraz ,
The compute needed for ingestion by the Eventstream (and thus the Eventhouse ingestion too) with put pressure on the Fabric capacity. Check the capacity size too.
An Eventstream is a combination of EventHubs and Azure Stream Analytics.
Because you are ingesting serious amounts of messages, you need to think about the quotas that these services have (lime the Eventhub).
I expect you are using the custom source as input for the Eventstream.
As an alternative, you can chose to ingest the messages first via an EventHub and make the Eventstream a consumer of the Eventhub. This gives much more control over the endpoint like scaling and pauzing egress.
If this answer helps you, a thumbs-up or marking it as accepted answer is appreaciated.
3 Replies
kustortininjaMicrosoft Employee
Hello, please start by checking the eventstream throughput settings on the Eventstream. Do you have it set appropriately?
Configure settings for a Fabric eventstream - Microsoft Fabric | Microsoft Learn
- svelde
Hello SavioFerraz ,
The compute needed for ingestion by the Eventstream (and thus the Eventhouse ingestion too) with put pressure on the Fabric capacity. Check the capacity size too.
An Eventstream is a combination of EventHubs and Azure Stream Analytics.
Because you are ingesting serious amounts of messages, you need to think about the quotas that these services have (lime the Eventhub).
I expect you are using the custom source as input for the Eventstream.
As an alternative, you can chose to ingest the messages first via an EventHub and make the Eventstream a consumer of the Eventhub. This gives much more control over the endpoint like scaling and pauzing egress.
If this answer helps you, a thumbs-up or marking it as accepted answer is appreaciated.
Hi SavioFerraz
Thank you for reaching out to the Microsoft Fabric Forum Community.
kustortininja svelde Thanks for your inputs
I hope the information provided by users was helpful. If you still have questions, please don't hesitate to reach out to the community.
