Databricks Queries NOT Hitting Databricks

Hi sarah_riecke,

Thank you for your suggestion; it’s an important consideration.

Currently, we haven’t set up any activities in the ADF pipeline to scale down or terminate the SQL Warehouse used by Power BI after the ELT process. The pipeline’s notebooks operate on different clusters, and the SQL Warehouse for reporting is separate from those resources.

Your point about warehouse availability during scaling is valid. Since the issue only happens during the early morning pipeline-triggered refresh and not during manual refreshes later, it’s possible the SQL Warehouse is in a stopped or starting state when Power BI tries to connect via the API.

No queries are logged in the SQL Warehouse monitoring during the failure, which suggests the connection attempt fails before reaching Databricks, possibly because the warehouse isn’t fully ready or responsive.

We will check:

If the SQL Warehouse is stopped before the refresh

Whether auto-stop or auto-start settings affect availability

Adding a pre-check step in ADF (such as a lightweight query or explicit start) before the Power BI refresh

Including a short delay after pipeline completion to ensure the warehouse is ready

This should help determine if the issue is related to warehouse readiness during the API-triggered refresh.

Thank you.