¿Is it possible to connect a broker (MQTT/Kafka) directly to Real-time Intelligence in MS Fabric?

[liney_marcela]

Hello community,

I am working on a project where I need to integrate a message broker (such as MQTT or Kafka) with the Real-time Intelligence service in MiS Fabric. Currently, I have managed to connect the broker to Fabric using a Python notebook, where the notebook receives data from the broker and ingests it into Fabric. However, this solution is not optimal because the notebook must be running all the time to receive the data, which is neither scalable nor efficient.

My questions are as follows:

Is there a native way to connect a message broker directly to Real-time Intelligence in Fabric, without relying on a notebook that is always running?

What additional tools or services could help me achieve this integration more efficiently?

What would be the most optimal and cost-effective solution for this use case? I am looking for an alternative that does not require keeping a notebook running constantly and can scale easily.

Additional context:

The broker is receiving real-time data from sensors.

Currently, the notebook acts as an intermediary to ingest the data into Fabric.

I am looking for a solution that is scalable, reliable, and minimizes operational costs.

I appreciate any guidance, documentation, or examples you can share to help resolve this issue. Thank you in advance for your help!

[Anonymous]

Hi @liney_marcela 

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[Anonymous]

Hi @liney_marcela 

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[Anonymous]

Hi @liney_marcela 
Thank you for reaching out microsoft fabric community forum.
I wanted to check if you had the opportunity to review the information provided @nilendraFabric . Please feel free to reach us if you have any further questions. If his response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.

[nilendraFabric · Accepted solution]

Hello @liney_marcela 

RTI supports native Kafka connectivity through its Eventstream custom endpoints

Create a Kafka-compatible custom endpoint in Fabric’s Eventstream interface using the “Custom Source” option
• Configure with Kafka protocol parameters

Bootstrap server: [uid].servicebus.windows.net:9093
Authentication: SASL/PLAIN with $ConnectionString username [1]
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/add-source-apache-kafk...

Or you can Use Azure IoT Operations’ dataflow engine to bridge MQTT brokers to Fabric RTI:

MQTT Source -> AIO Dataflow -> Fabric Eventstream [1]

if this is helpful please accept the answer