Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified as a Fabric Data Engineer: Check your eligibility for a 50% exam voucher offer and join us for free live learning sessions to get prepared for Exam DP-700. Get started

Reply
sarnendude
Frequent Visitor

High latency in Eventstream under 'Synapse Real-Time Analytics' in Microsoft Fabric

We created an Eventstream under 'synapse real-time analytics' in Microsoft Fabric to ingest real time events from Azure Event Hub to Fabric Lakehouse without any event processing by eventstream engine.
Multiple times we pushed events from Azure Event Hub using its 'Generate Data (Preview)' feature & it takes time between 30 seconds to 2 minutes to load into Fabric lakehouse.
Official documentaion says "....Eventstream with a latency of a few seconds".
In my scenario, variation of time 30 seconds to 2 minutes, so is it expected time to load or do we have any optimization steps to reduce this time? Please help.

Official documentation: https://learn.microsoft.com/en-us/fabric/real-time-analytics/overview

 

Thanks,

Sarnendu De

1 ACCEPTED SOLUTION
ajetasi
Microsoft Employee
Microsoft Employee

Currently, its a limitation that Eventstream to Lakehouse takes upto 2 minutes of latency. We are working on optimizing the latency by compacting small files that are generated in destination table. Please refer to lakehouse documentation under "Eventstreams"

 ajetasi_0-1691168381850.jpeg 

If you want a low latency (~2 seconds) destination from Eventstream, please feel free to use KQL DB destination. That is what is mentioned in the documentation you are referring to:

"You can stream large volumes of data into your KQL database through Eventstream with a latency of a few seconds, then use a KQL queryset to analyze"

 

View solution in original post

3 REPLIES 3
Tzvia
Microsoft Employee
Microsoft Employee

It sounds that If you need the lower latency from Ingestion to query, you should consider to use Synapse Real-Time Analytics at Microsoft Fabric that was built to support these scenarios 

https://learn.microsoft.com/en-us/fabric/real-time-analytics/

 

 

ajetasi
Microsoft Employee
Microsoft Employee

Currently, its a limitation that Eventstream to Lakehouse takes upto 2 minutes of latency. We are working on optimizing the latency by compacting small files that are generated in destination table. Please refer to lakehouse documentation under "Eventstreams"

 ajetasi_0-1691168381850.jpeg 

If you want a low latency (~2 seconds) destination from Eventstream, please feel free to use KQL DB destination. That is what is mentioned in the documentation you are referring to:

"You can stream large volumes of data into your KQL database through Eventstream with a latency of a few seconds, then use a KQL queryset to analyze"

 

Thank you for clarification & letting us know in details. It helps 😊

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Feb2025 Sticker Challenge

Join our Community Sticker Challenge 2025

If you love stickers, then you will definitely want to check out our Community Sticker Challenge!

JanFabricDE_carousel

Fabric Monthly Update - January 2025

Explore the power of Python Notebooks in Fabric!

Feb2025 NL Carousel

Fabric Community Update - February 2025

Find out what's new and trending in the Fabric community.

Top Solution Authors
Top Kudoed Authors