Databricks Queries NOT Hitting Databricks

Hi sarah_riecke,

Based on your description, it seems the scheduled refresh in Power BI is failing before the query reaches Databricks, which is why there’s no activity in the SQL Warehouse monitoring logs. Since the manual refresh works later in the day, the issue probably happens during the connection initialization from Power BI Service, not within Databricks.

This could be due to timing and overlapping workloads in the morning. When your ADF pipelines run to refresh Databricks and Power BI starts its dataset refresh around the same time, Power BI might have trouble connecting to the SQL Warehouse, especially if the warehouse needs to wake up or if other workloads are running. Power BI will retry several times, which matches the multiple retries and eventual failure you’ve noticed.

It’s also worth checking authentication and connection stability. If the Databricks connection uses a personal access token or stored credentials, there might be occasional authentication or connectivity issues during scheduled refresh. Re-validating credentials in the dataset settings can help rule this out.

Since the warehouse shows activity when you run the refresh manually, introducing a short delay between the ADF pipeline completion and the Power BI refresh could help, giving Databricks time to finalize updates and ensuring the warehouse is ready.

Reviewing the dataset refresh history in Power BI Service may also provide more details about the connection failure, such as activity IDs or error messages.

Overall, since the queries don’t appear in the Databricks logs, the issue is likely on the Power BI Service side during the connection stage, rather than with the warehouse configuration.

Thank you.