Spark Lib classpath not set - similar type of issue reported in 1085

[saravrathinam]

I am writing to seek your assistance regarding an issue with setting the Spark classpath for the Redshift JDBC driver (redshift-jdbc42-2.2.0).

Here are the steps I have followed so far:

1. I created an Environment, named it RedshiftLib, and then navigated to Custom Library. Here, I uploaded the file (redshift-jdbc42-2.2.0.jar), saved it, and published the changes.
2. After publishing, I created a new Notebook, attached the Environment to it, and started a new session.
3. In the Notebook, I executed the following command to validate whether the classpath was set:

```python

print("spark.jars =", spark.sparkContext.getConf().get("spark.jars",""))

cp = spark._jvm.java.lang.System.getProperty("java.class.path")

print("Found 'redshift' on classpath? ", "redshift" in cp.lower())

print(cp[:1200])

```

The result I received was:

```python

spark.jars =

Found 'redshift' on classpath? False

com.amazon.redshift.Driver (2.x): False

```

Alternatively, I loaded the file into the ABFS path and tried to register the path with the following command, but it still didn’t work:

```python

%%configure -f

{

  "jars": [   

    "abfss://xxxxxxxx@onelake.dfs.fabric.microsoft.com/xxxxxx.Lakehouse/Files/Common/Library/redshift-jdbc42-2.2.0.jar" 

  ]

}

```

After connecting the Environment in the Notebook, I removed the existing published library (redshift-jdbc42-2.2.0.jar), reloaded the same file, published it again, and tried to execute the validation command once more. Unfortunately, I am still encountering the same error—the classpath is not set.

This issue seems to be an extension of the known bug 1085. Could you kindly provide some inputs or guidance on how to resolve this problem?

Thank you for your time and support.

[v-tsaipranay]

Hi @saravrathinam ,

Thanks for reaching out to the Microsoft fabric community forum.

Please follow this steps which may help you resolve your issue:

• Upload your JAR to the Environment as described above (via the “Custom Library” section). Wait until publish completes.
• In the notebook, stop your current session and completely restart it (so Spark picks up the library at session start).
• Ensure the notebook is attached to the correct Environment (the one with the JAR).
• After the session starts, run your validation code to ensure the classpath includes the JAR.
• Then attempt a JDBC connection using the driver class name.

Hope this helps. Please feel free to reach out for any further questions.

Thank you.