Forum Discussion

yoshihirok's avatar
yoshihirok
Icon for Post Prodigy rankPost Prodigy
8 years ago
Solved

Can I connect Azure Databricks from Power BI Desktop by DirectQuery?

I want to know, Can I connect Azure Databricks from Power BI Desktop by DirectQuery?

Currently, the data sources for DirectQuery support Apache Spark on Azure HDInsight and Spark.

https://docs.microsoft.com/en-us/power-bi/desktop-directquery-data-sources

 

and, Power BI Desktop will support to analyze 1 trillion rows on Apache Spark.

https://www.microsoft.com/en-us/businessapplicationssummit/video/BAS2018-2172

 

and, Azure Databricks is Apache Spark-based analytics platform

https://azure.microsoft.com/en-us/services/databricks/

 

Therefore, How about Azure Databricks by DirectQuery?

What do you think?

 

Regards,

Yoshihiro Kawabata

 

Need Help
  • v-frfei-msft's avatar
    v-frfei-msft
    8 years ago

    Hi yoshihirok,

     

    Power BI Desktop can be connected directly to an Azure Databricks cluster using the built-in Spark connector (Currently in preview). The connector enables the use of DirectQuery to offload processing to Databricks. For more details, please check the online document.

     

    Regards,

    Frank

7 Replies

  • v-frfei-msft's avatar
    v-frfei-msft
    Icon for Community Support rankCommunity Support
    8 years ago

    Hi yoshihirok,

     

    Power BI Desktop can be connected directly to an Azure Databricks cluster using the built-in Spark connector (Currently in preview). The connector enables the use of DirectQuery to offload processing to Databricks. For more details, please check the online document.

     

    Regards,

    Frank

    • yoshihirok's avatar
      yoshihirok
      Icon for Post Prodigy rankPost Prodigy
      8 years ago

      Thank you v-frfei-msft for your nice information

       

      Awesome, Power BI Desktop can connect the data on Azure Databricks as DirectQuery

      Just, I'm looking the information for share with partners, friends.

       

      Yes, I will check the online document about Power BI Desktop and Azure Databriks.

      and, I hope to add the Azure Databricks to data source supported by DirectQuery.

       

      Regards,

      Yoshihiro Kawabata

    • Anonymous's avatar
      Anonymous
      Not applicable
      7 years ago

      The Spark connector is very slow....

       

      Queries in Azure Databricks are very fast less than 200ms, but data takes more than 10s to arrive to PowerBI Desktop...

       

      Can I do anything to optimize performance?

       

      Thank you,

      Miguel

      • Anonymous's avatar
        Anonymous
        Not applicable
        7 years ago

        This an example of the query on Dax Studio:

         

         

        This is the timings of the Azure Databricks server for the same querie:

         

         

         

         

        Can someone explain me why is so slow.....

         

        And its not a network problem, because if you run it from PowerBi Online it will take the same time...

         

        Thank you,

        Miguel

  • Anonymous's avatar
    Anonymous
    Not applicable
    6 years ago

    Hi,

     

    I followed the steps but no luck in connecting databricks from power BI.

    getting below error.


    Details: "ODBC: ERROR [HY000] [Microsoft][DriverSupport] (1170) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct."

     

    any help will be much appreciated.

    Thanks

    Jith

     

  • susriram's avatar
    susriram
    New Member
    7 years ago

    Yes, You can connect Azure Databricks from Power BI Desktop by DirectQuery. You need to select Spark connector. I tried in PowerBI November 2018 Version, it's working.

     

    FYI, BI Version that I used.

    Version: 2.64.5285.741 64-bit (November, 2018)