Forum Discussion

Kar_c's avatar
Kar_c
Regular Visitor
1 year ago

Near real time

Hi,   I have a requirement of pulling the data from an API in near real time(5 min interval) and load it into lakehouse or warehouse? How do I do it in fabric? The entire pipeline has to complete w...
Eventstream
lbendlin's avatar
lbendlin
Super User
1 year ago

I would have said "Use Eventhouses instead"  but these Kusto databases seem to have a horrible latency (8 minutes plus) - at least that's what the Capacity Metrics App exposes. So not sure if it would work in your scenario.

omarecd's avatar
omarecd
Advocate I
1 year ago

Hello lbendlin ,

 

I was reading and I find interesting your feedback. I have been doing some test from my side and the Kusto Database performs actually very correct, a couple of seconds of delay is more than enough for me. Would it be possible to share a screenshot or extra details about the latency that you mention ? Unfortunately I can't use/install the Capacity Metrics App because I am missing some rights...

 

Thanks and regards,

Omar C.

  • lbendlin's avatar
    lbendlin
    Super User
    1 year ago

    I believe the difference is in the configuration - batch mode vs streaming mode.  The Capacity Metrics app source seems to be running in batch mode.