Forum Discussion
Super UserConnection with databrick detla tables
Hi everyone,
I am trying to connect Power BI to Databricks Delta tables. The connection works, but the queries feel slow when refreshing visuals and running transformations.
Has anyone optimized this setup before?
What connection mode and configuration worked best for you?
Key details:
Data source: Databricks Delta tables
Destination: Power BI
Looking for practical steps or best practices that help speed things up.
Hey Shubham_rai955 ,
I assume you use Direct Query. Direct Query is never speedy unless your data is tiny. I generally use Databricks with Import, the load itself it quite fast, but it depends on your cluster, etc.Hi Shubham_rai955,
Here is some best practices that help speed things up:
- Switch to a SQL Warehouse
- Do not use a
classic clusterfor BI, SQL Warehouses (especially serverless) are built for this and are much faster
- Do not use a
- Use Import Mode, Not DirectQuery
- If your dataset fits in memory, Import mode is almost always faster; You trade off some data latency for much snappier report interaction
- Optimize Your Delta Tables
- This is the biggest win. Run these commands on your big tables in Databricks:
OPTIMIZE your_table ZORDER BY (your_common_filter_column, date_column);This groups data together so Databricks can skip files when querying
- Build a Star Schema in Power BI
- Flattened tables are easier to build but slower to query, Create a proper star schema with separate fact and dimension tables
- Use the Latest Connector
- Make sure you are using the current Databricks connector in Power BI (The older ODBC driver is slower)
if this post helps, then I would appreciate a thumbs up and mark it as the solution to help the other members find it more quickly.- Switch to a SQL Warehouse
4 Replies
- vojtechsima
Hey Shubham_rai955 ,
I assume you use Direct Query. Direct Query is never speedy unless your data is tiny. I generally use Databricks with Import, the load itself it quite fast, but it depends on your cluster, etc. 
v-echaithraCommunity Support
Hi Shubham_rai955 ,
Just following up to see if the Response provided by community members were helpful in addressing the issue. if the issue still persists Feel free to reach out if you need any further clarification or assistance.
Best regards,
Chaithra E.- v-echaithra
Hi Shubham_rai955 ,
May I ask if you have resolved this issue? Please let us know if you have any further issues, we are happy to help.Thank you.
- Ahmed-Elfeel
Hi Shubham_rai955,
Here is some best practices that help speed things up:
- Switch to a SQL Warehouse
- Do not use a
classic clusterfor BI, SQL Warehouses (especially serverless) are built for this and are much faster
- Do not use a
- Use Import Mode, Not DirectQuery
- If your dataset fits in memory, Import mode is almost always faster; You trade off some data latency for much snappier report interaction
- Optimize Your Delta Tables
- This is the biggest win. Run these commands on your big tables in Databricks:
OPTIMIZE your_table ZORDER BY (your_common_filter_column, date_column);This groups data together so Databricks can skip files when querying
- Build a Star Schema in Power BI
- Flattened tables are easier to build but slower to query, Create a proper star schema with separate fact and dimension tables
- Use the Latest Connector
- Make sure you are using the current Databricks connector in Power BI (The older ODBC driver is slower)
if this post helps, then I would appreciate a thumbs up and mark it as the solution to help the other members find it more quickly. - Switch to a SQL Warehouse