Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

VS Code notebook failure - Spark

I want to integrate VS code with my Fabric workspace and I have been able to successfully do it. I have followed all the pre-requisites mentioned in this link - https://learn.microsoft.com/en-us/fabric/data-engineering/setup-vs-code-extension.

 

But when I try to run any spark related commands in VS code, I get an error - "AttributeError: 'NoneType' object has no attribute 'sql'" and "spark is not defined Pylance". However these commands are successfully running in Fabric.

 

My kernel specification is - fabric-synapse-runtime-1-2(Python 3.10.12)

 

As a quick fix, I tried to reinstall VS Code and the extension but unfortunately nothing worked. 
Would appreciate any quick help or solution on this.

 

Thanks.

data engineering
Notebook
  • Anonymous's avatar
    Anonymous
    2 years ago

    The solution that worked for me is - Re-installing VS Code by clearing the cache files as well during uninstallation.

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    The solution that worked for me is - Re-installing VS Code by clearing the cache files as well during uninstallation.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Anonymous 
      Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Anonymous 

    Thanks for using Fabric Community.
    Please follow these steps:
    1) Right click the ipython notebook

     

     



    Reveal in File Explorer

    2) Open the python log file: ../../../logs/${NOTEBOOK_ARTIFACT_ID}/PySparkLighter.log and check the session failure reason

     

    Please attach the screenshot of the error here to understand what is happening.

    You can also try stopping the spark session in the Notebook and try running in VS code. 
    Thanks

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Anonymous, 

      Here is the screenshot of the logs that i can see.

       

      Also how do I stop the Spark session in notebook and run in VS code?

       

      Thanks.