Forum Discussion
Near real time
Hi, I have a requirement of pulling the data from an API in near real time(5 min interval) and load it into lakehouse or warehouse? How do I do it in fabric? The entire pipeline has to complete w...
You can use the Notebook as a streaming notebook (with a wait command for 5 mins) and send the data to an Eventstream.
Here you can transform the data (unfold the json) quite easily and land it in a Lakehouse and/or Eventhouse to fit your needs for further processing of the data.
The latency mentioned above is only for the mirror between Eventhouse and OneLake - but you can land your data directly in a Lakehouse from the Eventstream with extremely low latency.