Forum Discussion

joshua1990's avatar
joshua1990
Icon for Post Prodigy rankPost Prodigy
4 years ago
Solved

Difference between Spark and Databricks Dataflow

Hi experts! I recently noticed that there is now a Databricks (Beta) connector for the Dataflows > Awesome! In the past I always used Spark to connect to Databricks. But now, is there any advantag...
Need Help
  • ibarrau's avatar
    ibarrau
    4 years ago

    I mean the main processor at databricks. The engine you need to run in order to use it. The name it's cluster on a Data engineer or Science environemnt or you can also find it as SQL Endpoint or SQL Warehouse at a SQL environment.

    Power Bi gets data from the metastore of databricks, you need .... "a cluster" turned on to read it.

    I hope that make sense I can add pictures if it's still not clear for you.

joshua1990's avatar
joshua1990
Icon for Post Prodigy rankPost Prodigy
4 years ago

Thanks a lot! What do you mean with Cluster in this conext?

ibarrau's avatar
ibarrau
Icon for Super User rankSuper User
4 years ago

I mean the main processor at databricks. The engine you need to run in order to use it. The name it's cluster on a Data engineer or Science environemnt or you can also find it as SQL Endpoint or SQL Warehouse at a SQL environment.

Power Bi gets data from the metastore of databricks, you need .... "a cluster" turned on to read it.

I hope that make sense I can add pictures if it's still not clear for you.