The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Hello community,
I am working on a project where I need to integrate a message broker (such as MQTT or Kafka) with the Real-time Intelligence service in MiS Fabric. Currently, I have managed to connect the broker to Fabric using a Python notebook, where the notebook receives data from the broker and ingests it into Fabric. However, this solution is not optimal because the notebook must be running all the time to receive the data, which is neither scalable nor efficient.
My questions are as follows:
Is there a native way to connect a message broker directly to Real-time Intelligence in Fabric, without relying on a notebook that is always running?
What additional tools or services could help me achieve this integration more efficiently?
What would be the most optimal and cost-effective solution for this use case? I am looking for an alternative that does not require keeping a notebook running constantly and can scale easily.
Additional context:
The broker is receiving real-time data from sensors.
Currently, the notebook acts as an intermediary to ingest the data into Fabric.
I am looking for a solution that is scalable, reliable, and minimizes operational costs.
I appreciate any guidance, documentation, or examples you can share to help resolve this issue. Thank you in advance for your help!
Solved! Go to Solution.
Hello @liney_marcela
RTI supports native Kafka connectivity through its Eventstream custom endpoints
Create a Kafka-compatible custom endpoint in Fabric’s Eventstream interface using the “Custom Source” option
• Configure with Kafka protocol parameters
Bootstrap server: [uid].servicebus.windows.net:9093
Authentication: SASL/PLAIN with $ConnectionString username [1]
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/add-source-apache-kafk...
Or you can Use Azure IoT Operations’ dataflow engine to bridge MQTT brokers to Fabric RTI:
MQTT Source -> AIO Dataflow -> Fabric Eventstream [1]
if this is helpful please accept the answer
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
Hi @liney_marcela
Thank you for reaching out microsoft fabric community forum.
I wanted to check if you had the opportunity to review the information provided @nilendraFabric . Please feel free to reach us if you have any further questions. If his response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.
Hello @liney_marcela
RTI supports native Kafka connectivity through its Eventstream custom endpoints
Create a Kafka-compatible custom endpoint in Fabric’s Eventstream interface using the “Custom Source” option
• Configure with Kafka protocol parameters
Bootstrap server: [uid].servicebus.windows.net:9093
Authentication: SASL/PLAIN with $ConnectionString username [1]
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/add-source-apache-kafk...
Or you can Use Azure IoT Operations’ dataflow engine to bridge MQTT brokers to Fabric RTI:
MQTT Source -> AIO Dataflow -> Fabric Eventstream [1]
if this is helpful please accept the answer