Anyone else having random errors show up in notebooks?

Hello arpost 

Thank you for reaching out to Microsoft Community Forum.

The error message

Py4JJavaError: An error occurred while calling...java.lang.IllegalStateException: SparkSession should only be created and accessed on the driver usually indicates an attempt to create or access a SparkSession on a worker node instead of the driver node.

1. Ensure the SparkSession is created only once and on the driver node. Avoid creating it inside functions executed on worker nodes.
   
   
2. Verify that code involving SparkSession operations is not executed within transformations or actions on worker nodes, such as map, filter, or other RDD operations.
   
   
3. Ensure the SparkSession is not referenced as a global variable inside functions on worker nodes.
   

If the issue persists, consider sharing the specific code snippet causing the error for a detailed analysis.

If my response has resolved your query, please mark it as the Accepted Solution to assist others. A 'Kudos' would also be appreciated if you found my response helpful.

Thank you!