Forum Discussion

Tom117's avatar
Tom117
Frequent Visitor
2 years ago

spark not defined using VSCode Synapse Extension

I am attempting to work on a pyspark notebook locally using VS Code and the Microsoft Synapse extension.   I installed all dependencies as described in the documentation for the extension. I've con...
workspaces
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi  Tom117 

 

After some time I finally got it work at my end. Below are what I have done before it works. Not sure which step is the key but you may have a try!

 

1. Use Synapse VS Code extension v1.9.0. This version fixes an issue related to the default lakehouse of notebook, which occurs in v1.8.0. 

 

 

2. Delete the notebook from your local drive. Delete the notebook folder in your Local Work Folder Path.

 

 

3. Sign off your account from Synapse VS Code extension. 

 

4. On Jupyter extension settings page, extend Jupyter Interrupt Timeout and Jupyter Launch Timeout. I set 100000 and 600000 for them. 

 

5. Close VS Code and restart it. Sign in acount to Synapse VS Code extension.

 

6. Download the notebook to local drive again. This will re-generate a configure file for the notebook. 

 

7. Open the notebook and run a code cell. Although it shows "spark is not defined", it run successfully as a result. 

 

Hope this would be helpful. 

 

Best Regards,
Jing
If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!