Forum Discussion

paatrickd's avatar
paatrickd
New Member
2 years ago
Solved

VS Code Synapse integration not working

Hi,   I want to open a notebook from Fabric in Visual Studio Code.   I followed the installation guide and managed to activate the Synapse Kernal and open a notebook in VS Code, however it does n...
data engineering
data science
data warehouse
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi paatrickd 
    Thanks for using Fabric Community.
    Can you please follow the below troubleshooting steps and provide the details:

    1) Right click the ipython notebook.

     




    2) Open the python log file: ../../logs/${NOTEBOOK_ARTIFACT_ID}/PySparkLighter.log and check the session failure reason.


    Please provide the error details so that I can help you better.

    Thank you.

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi paatrickd 
    Thanks for the update.
    Maybe you have installed conda, but forgot to set the environment variable. Please check that.
    After creating the conda environment, it will install the spark-lighter-lib whl package from blob. The whl package will be automatically upgraded when the version in the conda environment does not match the minimum required version of the extension. If there is any issue with the upgration, you could run the below code :

    conda activate ${CONDA_ENV_NAME}
pip uninstall spark-lighter-lib
pip install https://path/to/spark-lighter-lib.whl


    Also try these steps.

    1. Open the VS Code command palette and select/type 'Python: Create Terminal'
    2. In the terminal, run the following command: conda install pyspark
    3. Restart VS Code and try again.


    Hope this helps. Please let me know if you have any further questions.

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi paatrickd ,
    Go to the upper folder to find a folder called logs.

     

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi akanane ,
Can you please check the syntax of JAVA_HOME environment variable?

Please change it to %JAVA_HOME%.

Hope this helps. Please let me know if you have any further questions. Glad to help.

akanane's avatar
akanane
Frequent Visitor
2 years ago

Hello Anonymous, thanks for point it out the typo.. I fixed i but still cannot use spark, this is the error code now:
09:42:41,388 root INFO current_directory c:\Users\akanane\AppData\Local\miniconda3\772d3ce8-1d2e-4d53-b848-97351ede78a3\SynapseNotebook\4627fe3a-a5dc-4ff7-b3f5-3067b88b080f\BuildFactSales
09:42:41,388 root INFO workspace_path c:\Users\akanane\AppData\Local\miniconda3\772d3ce8-1d2e-4d53-b848-97351ede78a3
09:42:41,388 root INFO log_path c:\Users\akanane\AppData\Local\miniconda3\772d3ce8-1d2e-4d53-b848-97351ede78a3\logs\4627fe3a-a5dc-4ff7-b3f5-3067b88b080f
09:42:41,388 root INFO Using synapse remote kernel ...
09:42:41,388 root INFO Should attach session in dev mode False
09:42:41,468 root INFO Starting session b60ba0ac-6d16-4341-9ae9-f96f55ae67ab...
09:42:43,629 root INFO https://pbipwus13-westus.pbidedicated.windows.net/webapi/capacities/07652219-C048-4E90-B99A-868EC7DCB544/workloads/Notebook/Data/Direct/sparklighter/api/tjs/versions/2022-04-30/artifacts/4627fe3a-a5dc-4ff7-b3f5-3067b88b080f/sessions/b60ba0ac-6d16-4341-9ae9-f96f55ae67ab
09:42:45,71 root INFO <session_management.SessionStatus object at 0x0000019A9A936800>
09:42:45,79 root ERROR Failed to initialize Spark Lighter variables. Create session failed {'code': 'InternalError', 'subCode': 0, 'message': 'An internal error occurred.', 'timeStamp': '2023-11-02T08:42:44.9891628Z', 'httpStatusCode': 500, 'hresult': -2147467259, 'details': [{'code': 'RootActivityId', 'message': 'a038d8a0-e2e4-4375-a8ed-3dfb9866d56f'}, {'code': 'Param1', 'message': 'Response status code does not indicate success: 430 ().'}]}
09:42:45,80 root INFO Registering Spark Lighter magics for IPython...
09:42:45,80 root INFO Registered Spark Lighter magics for IPython.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi akanane 
    You are facing this error because all the available capacity is currently being used. Cancel some currently running Notebooks or Spark Job Definition jobs, increase your available capacity and then try to rerun the notebook.

    If the issue still persists please let me know.
    Thanks.