Forum Discussion

jjaeger94's avatar
jjaeger94
Helper II
2 years ago
Solved

Error Connecting to Delta Share

I get the following error when trying to connect to a Databricks Delta Share using a Notebook:   Failed to find the data source: deltaSharing. Please find packages at `https://spark.apache.org/thir...
data engineering
notebook
Spark Job Definition
  • jjaeger94's avatar
    jjaeger94
    1 year ago

    The solution was to set up an environment with the delta_sharing library and the following Spark configuration.

     

    runtime_version: '1.3'
    spark_conf:
    - spark.jars.packages: >-
    org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.2.0,io.delta:delta-sharing-spark_2.12:3.1.0
    - spark.sql.extensions: io.delta.sql.DeltaSparkSessionExtension
    - spark.sql.catalog.spark_catalog: org.apache.spark.sql.delta.catalog.DeltaCatalog

jjaeger94's avatar
jjaeger94
Helper II
2 years ago

Thank you for reaching out.

 

Here is the code:

 

 

 

Here is the error:

 

 

The following code loading as a pandas dataframe works, so I know the connection to the config file is not a problem and that the table exists:

 

 

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi jjaeger94 

The best course of action is to open a support ticket and have our support team take a closer look at it. Please reach out to our support team using this: https://support.fabric.microsoft.com/en-IN/support/ so they can conduct a more thorough investigation into why this is happening.

After creating a support ticket, please provide the ticket number to help us track and gather more information.

Hope this helps. Please let us know if you have any other queries.