Forum Discussion
PBI PPU Import Dataset from Databricks SQL Warehouse Timeout
Hi SethParker
Could I ask why you are doing the full refresh every night, if you have got incremental refreshing working that should always get the latest data?
Also there is no way for Power BI to know that the query has failed from databricks.
Hello Gilbert,
Thank you for your answer. We could just run the incrementals, but due to the dynamic nature of our partitions, we would have to rerun the full load at least weekly. This is irrelevant to the underlying issue, though.
Power BI does know when Databricks queries fail for other reasons. For example, I wrote a view that has a datatype cast issue (as a test). The dataset refresh returned the following error when it tried to load that table:
Data source error: {"error": {"code": "SSAS_ShortMessage_ProcessingError","pbi.error": {"code": "SSAS_ShortMessage_ProcessingError","parameters": {},"details": [],"exceptionCulprit": 1}}} Table: GL Balances FailTest.
It did not just sit there waiting for results for 5 hours. But when the Databricks SQL query fails with the "Query timed out due to inactivity" error, it does sit until the 5 hour timeout.