Real-time streaming in Fabric Power BI is not working as expected

[Kuladeep]

Currently, I am working on a project to analyse real-time data coming from a Kafka-enabled Event Hub. I have accomplished almost everything but stumbled at the final step.

What I’ve done so far:

• Created an Azure VM.
• Developed a Python script to run Event Hub on the VM and produce data using the Faker package every 3 seconds (see code below).
• Connected the Event Hub to Fabric Eventstream.
• Configured the Eventstream destination to a Lakehouse Delta table (I could not create a KQL database. Only available KQL query dataset option). The destination creates a new file either after 1,000 rows or 120 seconds. Dimension tables were already loaded from files.
• Established entity relationships between the tables in the auto-generated semantic model. I also checked the refresh settings in the semantic model’s settings
• Created a card visual (total revenue) in Power BI based on the semantic model to verify real-time updates. I pinned this visual to a dashboard as well. However, it does not update unless I manually press the refresh button.

I hope I have followed the correct procedure. Can someone please help me to achieve a real time data visualization as shown in Real-time streaming in Power BI

Workspace:

Semantic model:

Eventstream:

PowerBi dasboard:

[Kuladeep]

Thanks @v-nmadadi-msft 

One last question why can't I see create KQL database option in my Fabric free capacity? Is that needs to activated from admin portal or its been in developement/maintanance?

[v-nmadadi-msft · Accepted solution]

Hi @Kuladeep ,
Thanks for reaching out to the Microsoft fabric community forum.

To create a KQL database the prerequisite is to have a workspace in a fabric enabled capacity.

I hope this information helps. Please do let us know if you have any further queries.
Thank you

[v-nmadadi-msft]

Hi @Kuladeep ,
Thanks for reaching out to the Microsoft fabric community forum.

As mentioned by @uzuntasgokberk  , To display the dashboard accurately in Power BI, you need to configure a scheduled refresh. Please note that in shared capacity, semantic models are limited to eight scheduled refreshes per day, but manually you can manually refresh more than 8 times a day if your dataset is in shared capacity. 
Data refresh in Power BI - Power BI | Microsoft Learn

If you want something that is real time you might have to configure a real time dashboard for which you would require KQL database,
Please follow the below article to know how to create a real time dashboard:
Create a Real-Time Dashboard - Microsoft Fabric | Microsoft Learn

I hope this information helps. Please do let us know if you have any further queries.
Thank you

[uzuntasgokberk]

Hello @Kuladeep ,

If you prefer to continue using a Power BI Dashboard without relying on custom streaming data(pubnub, azure stream, api), the semantic model must be either refreshed at high frequency with İmport mode(with / without İncremental Refresh) or configured with Direct Lake or DirectQuery(I prefer to auto refresh setting open in PBI.).

However, my recommendation is to use the Real-Time Intelligence Dashboard, as it offers true real-time data visualization, making it more suitable for immediate insights especially with KQL Database or Azure Data Explorer.

Kind Regards,
Gökberk Uzuntaş

📌 If this post helps, then please consider Accepting it as a solution and giving Kudos — it helps other members find answers faster!

🔗 Stay Connected:
📘 Medium |
📺 YouTube |
💼 LinkedIn |
📷 Instagram |
🐦 X |
👽 Reddit |
🌐 Website |
🎵 TikTok |

[Kuladeep]

Thanks for reply @uzuntasgokberk .

However since I am new to Fabric, I have couple of noob questions. I appreciate your response

1. Since I’m using Eventstream in Fabric, where can I find sources like PubNub, Azure Stream, or custom APIs? Aren’t these specific to Power BI Premium outside of Fabric? Eventstream connected to Event hubs is a kind of Azure stream, right?

2. Since I am on Fabric trail, I can only use Directquery. But where can I find/activate the option in powerbi while connecting to semantic model?
3. what do you mean by  auto refresh setting open in PBI? - Does that mean schedule refresh configured on semantic model or there is any other way of refreshing the dashboard?
4. Unfortunately, I am unable to activate my Real-Time Intelligence Dashboard in Fabric, even though I am the sole owner of my account. I have already raised a support ticket to inquire why I’m unable to access the admin settings. Am I unable to use a KQL database unless this feature has been enabled at the admin level?

Finally, what possible solutions do I have in this scenario—without access to KQL, Direct Lake, and with a faulty admin portal—to create a Real-Time Intelligence Dashboard?

[v-nmadadi-msft]

Hi @Kuladeep ,
Thanks for reaching out to the Microsoft fabric community forum.

1) These are some of the supported data sources in eventstream: Azure Data Explorer, Azure Event Hubs, Azure Event Grid, Azure Service Bus, Azure IoT Hub, Custom endpoint.
Microsoft Fabric event streams overview - Microsoft Fabric | Microsoft Learn

2) Please follow this article to know how to setup schedule refresh:
Configure scheduled refresh - Power BI | Microsoft Learn

3) Schedule refresh needs to be configured on semantic model.

I hope this information helps. Please do let us know if you have any further queries.
Thank you