Forum Discussion

jjaeger94's avatar
jjaeger94
Helper II
2 years ago
Solved

Error Connecting to Delta Share

I get the following error when trying to connect to a Databricks Delta Share using a Notebook:   Failed to find the data source: deltaSharing. Please find packages at `https://spark.apache.org/thir...
  • jjaeger94's avatar
    1 year ago

    The solution was to set up an environment with the delta_sharing library and the following Spark configuration.

     

    runtime_version: '1.3'
    spark_conf:
    - spark.jars.packages: >-
    org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.2.0,io.delta:delta-sharing-spark_2.12:3.1.0
    - spark.sql.extensions: io.delta.sql.DeltaSparkSessionExtension
    - spark.sql.catalog.spark_catalog: org.apache.spark.sql.delta.catalog.DeltaCatalog