Forum Discussion
spark not defined using VSCode Synapse Extension
Thank you so much for the suggestion here. I really appreciate your time. However, I gave that solution a try with no success 😕
Hi Tom117
I’m sorry to hear that the solution I provided did not bring a success. I understand how frustrating this can be. Let's further analyze this issue.
1. Check whether the configuration information of the notebook is generated correctly. You can find a configuration file via path like <your Local Work Folder path>\<workspace id>\SynapseNotebook\<notebook id>\conf\lighter-config.json. It's in the following format, ensure the information is correct. Note: Do not share the file content because it includes sensitive data.
2. Check whether the session started successfully. You can find two log files in the path <your Local Work Folder path>\<workspace id>\logs\<notebook id>.
Open the log files to check if there is any error occurring. If a session started successfully, you will see something like below.
PySparkLighter.log
SparkLight.log
If the session didn't start successfully, did it return any error message? Can you share the error message if possible? Please remove sensitive data.
Best Regards,
Jing
Hey, thank you very much for getting back to me!
So, my config looks just about the same with the exception of the "auto_session" and "session_id" properties. They do not exist in my config.
Additionally, I looked for the log files, however they also do not exist.
Any idea why that might be the case? Perhaps my instillation has never attempted to connect to the remote?Hi Tom117
It seems the remote session has not been triggered or started successfully. Can you check the versions of the prerequisite installations, especially the version of the jdk? From this thread, it appears openjdk21 is not supported and openjdk8 is required. And ensure the jdk bin and conda bin paths are added to the Path environment variable on your system. You could add them to the Path environment variable for both user variables and system variables at the same time.
Then restart your computer, open VS Code, delete the notebook from local and from the Local Work Folder. Download a notebook again, open it and wait a while for the kernel to be ready. After that, try running a code cell to start the remote session.
Best Regards,
Jing