topic Re: Architecture question and brainstorming in Real-Time Intelligence

Architecture question and brainstorming

Ira_27 — Fri, 20 Mar 2026 18:39:13 GMT

Hello Community,

I have a design question that I need help with, we have a lot of vendor data that comes in form of CSV, XML, XLSX files via sftp. I already have ADF that copies this files into ADLS Gen2. Now when implementing in fabric I actually used blob storage events to trigger a notebook to load this files into a Lakehouse. I want to get suggestions if it's a good idea to use real time intelligence to stream data from this files and stage into a lakehouse? Has anyone implemented this? If so would love to get guidance. I am fairly new to evetstream and need more details on CU consumption, cost etc. what are disadvantages of using streaming notebook?

Re: Architecture question and brainstorming

v-veshwara-msft — Mon, 23 Mar 2026 05:27:27 GMT

Hi Ira_27 ,

Thanks for raising this in Microsoft Fabric Community.

Eventstream is primarily intended for real-time ingestion from sources like Event Hub, IoT Hub, or Kafka. In your case, since vendor data arrives as batch files via SFTP and is already being copied into ADLS, using Eventstream directly on those files is not the typical approach. This is because Eventstream operates on streaming records rather than directly reading and processing files from storage.

Microsoft Fabric Eventstreams Overview - Microsoft Fabric | Microsoft Learn

The blob-triggered notebook pattern you mentioned is a good fit for this scenario, as it processes files as they arrive and loads them into the Lakehouse efficiently. This aligns well with common batch or micro-batch ingestion patterns.

If you want to explore Real-Time Intelligence, you would first need to convert those files into events, for example by pushing rows into Event Hub or another streaming source, and then let Eventstream process them. This adds additional complexity and overhead without a clear benefit for file-based ingestion.

From a cost perspective, Eventstream typically runs continuously for streaming workloads and consumes capacity based on runtime and data throughput. Similarly, streaming notebooks remain active and consume compute even when data volume is low, which can make them more expensive and harder to manage compared to batch processing.

Microsoft Fabric Eventstreams Capacity Consumption - Microsoft Fabric | Microsoft Learn

Understanding CU consumption of streaming systems in Microsoft Fabric

For vendor data that is inherently batch-oriented, an event-driven batch approach (like your current design) is usually simpler, more cost-effective, and easier to maintain. Eventstream is better suited for scenarios where data is naturally generated as real-time events and requires low-latency processing.

Hope this helps. Please reach out for further assistance.
Thank you.

Re: Architecture question and brainstorming

tayloramy — Mon, 23 Mar 2026 13:40:32 GMT

Hi Ira_27,

I would not recommend using RTI for anything that isn't real time, while you probably could use RTI for batch based loads, it would be highly inefficient as the RTI processes would be running 24/7 looking for data.

Re: Architecture question and brainstorming

Ira_27 — Thu, 26 Mar 2026 02:05:21 GMT

Thank you for your response. I am aligned with your suggestion but still wanted to get a confirmation from the community. Appreciate your response.

Re: Architecture question and brainstorming

Ira_27 — Thu, 26 Mar 2026 02:06:52 GMT

Appreciate your detailed response on it and thank you for confirming my current architecture.