Real Time Dashboard

Azure Databricks is not suitable for this.  Look into Eventhouses and how to feed data into them in real time.  Then you can use an actual dashboard to display the live data.

For reports you can use a Direct Query connection with Automatic Page Refresh, but that must be enabled on tenant level and your tenant admin will likely restrict you to a much higher sampling rate (more like 60 seconds or higher)

ShivaPr 

https://youtu.be/TwMBpV_7V-A

https://youtu.be/-nx3Ax7lNwA

If this answer helped, please click 👍 or Accept as Solution.
-Kedar
LinkedIn: https://www.linkedin.com/in/kedar-pande

Hi ShivaPr 

Connect Power BI to Databricks - https://youtube.com/playlist?list=PLE_IpfSmEJ83xroWa2oN5RI5qzBnwS5KW&si=YKG20oZZNbgkNbO-

Option 1 - Near Real-Time Report

Best for interactive reports
Architecture
Databricks → SQL Warehouse → Power BI DirectQuery

Steps
In Databricks, expose data via Databricks SQL Warehouse
In Power BI Desktop
>Get Data → Azure Databricks
>Choose DirectQuery
Optimize model
>Star schema
>Fewer columns
>Avoid complex DAX
Publish report
In Power BI Service
>Enable Auto Page Refresh
>Set refresh interval to 30 seconds (minimum allowed)

Limitations
5 seconds
Heavy visuals will slow down
DAX must be simple

Option 2 - True Real-Time (5 seconds) – Dashboard Only

Best for KPIs & monitoring
Architecture
Databricks → Event Stream → Power BI Streaming Dataset → Dashboard

Steps
Push data from Databricks to:
>Azure Event Hub / REST API
Create Streaming Dataset in Power BI Service
Build Dashboard tiles (not reports)
Tiles update every 1–5 seconds

Limitations
No report visuals
No complex modeling
No relationships

A true “every 5 seconds” real-time dashboard is not realistic in Power BI with Databricks, but you have a few near-real-time options.

Option 1: DirectQuery to Azure Databricks

This is the closest you’ll get.

How it works:

• Use DirectQuery connection to Databricks

• Visuals re-query the source on interaction

• Data is as fresh as Databricks at query time

Limits to know:

• Power BI auto page refresh minimum is 30 seconds (with Premium / Fabric)

• On Pro/shared capacity, auto page refresh is very limited or unavailable

• Heavy dashboards will struggle if queries are complex

This gives near-real-time, not 5 seconds.

Option 2: Direct Lake (if you’re on Fabric)

If Databricks writes to OneLake / Delta tables:

• Use Direct Lake semantic model

• No import refresh needed

• Very fast query performance

• New data becomes visible as files land

Still:

• Visual refresh is not every 5 seconds automatically

• Best for high-volume, low-latency analytics

Option 3: Streaming / Push datasets

Power BI streaming datasets can update in seconds, BUT:

• They don’t work with Databricks directly

• Limited visuals

• No complex DAX / relationships

Usually not suitable for enterprise dashboards.