Forum Discussion

jjaeger94's avatar
jjaeger94
Icon for Helper II rankHelper II
2 years ago
Solved

Error Connecting to Delta Share

I get the following error when trying to connect to a Databricks Delta Share using a Notebook:   Failed to find the data source: deltaSharing. Please find packages at `https://spark.apache.org/thir...
data engineering
notebook
Spark Job Definition
  • jjaeger94's avatar
    jjaeger94
    1 year ago

    The solution was to set up an environment with the delta_sharing library and the following Spark configuration.

     

    runtime_version: '1.3'
    spark_conf:
    - spark.jars.packages: >-
    org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.2.0,io.delta:delta-sharing-spark_2.12:3.1.0
    - spark.sql.extensions: io.delta.sql.DeltaSparkSessionExtension
    - spark.sql.catalog.spark_catalog: org.apache.spark.sql.delta.catalog.DeltaCatalog

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi jjaeger94 
Thanks for using Fabric Community.
Can you please send me the screenshot of the error along with the complete code? 

Thanks

jjaeger94's avatar
jjaeger94
Icon for Helper II rankHelper II
2 years ago

Thank you for reaching out.

 

Here is the code:

 

 

 

Here is the error:

 

 

The following code loading as a pandas dataframe works, so I know the connection to the config file is not a problem and that the table exists:

 

 

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi jjaeger94 

    The best course of action is to open a support ticket and have our support team take a closer look at it. Please reach out to our support team using this: https://support.fabric.microsoft.com/en-IN/support/ so they can conduct a more thorough investigation into why this is happening.

    After creating a support ticket, please provide the ticket number to help us track and gather more information.

    Hope this helps. Please let us know if you have any other queries.