Forum Discussion
Graphframes error: ClassNotFoundException
Anonymous : I received a message from your support engineer with the following solutions:
- Use Scala instead of Python:
- Add the library manually to the environment:
- Install the jar from maven repository. reference: Maven Repository: graphframes » graphframes » 0.8.0-spark3.0-s_2.12 (mvnrepository.com)
- upload the Jar to your environment. Reference: Library management in Fabric environments - Microsoft Fabric | Microsoft Learn
- Attach the environment to your workspace. Reference: Create, configure, and use an environment in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
- Add this config as first cell inside your notebook:
%%configure -f { "conf": { "spark.jars.packages": "graphframes:graphframes:0.8.0-spark3.0-s_2.12" } }This solved the issue.
Thanks! Avi
Hi goreavin ,
Thanks for using Fabric Community. I apologize for the issue you are facing.
These are the few steps to resolve your issue:
1. Include GraphFrames in Spark Session Configuration:
Even though graphframes is installed in your conda environment, it might not be included in the Spark session. You need to explicitly tell Spark to use it. Here's how:
from pyspark.sql import SparkSession
spark = SparkSession.builder \
.master("local[*]") \
.config("spark.jars.packages", "graphframes:graphframes:0.6") \
.getOrCreate()
# Now you can create your GraphFrame
g = GraphFrame(v, edge)
Explanation:
spark.jars.packages: This configuration tells Spark to include the JAR file for graphframes (version 0.6 in this case) when launching the Spark application.
2. Verify Spark Version Compatibility:
Double-check the compatibility between your Spark version and the installed graphframes version (0.6). Refer to the GraphFrames documentation for supported Spark versions for your specific graphframes version - link. There might be a mismatch causing the class not to be found.
3. Reinstall or Update GraphFrames (if necessary):
If the above steps don't work, consider reinstalling graphframes.
By following these steps, you should be able to resolve the ClassNotFoundException.
Hope this helps you in resolving the issue. Please let me know incase if you have further queries.
Hi goreavin ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .
Hi goreavin ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .