topic Re: VS Code and Synapse Kernel in Data Engineering

VS Code and Synapse Kernel

DennesTorres — Tue, 24 Oct 2023 16:34:46 GMT

Hi,

I found on the documentation that I should run Fabric notebooks using the Synapse Kernel.

However, the Synapse Kernel doesn't appear for me.

I installed java 1.8 and miniconda, I installed Synapse VS Code Extension (image below), I can see the workspaces and lakehouses, but the kernel is not available to run the notebooks.

What am I missing?

Kind Regards,

Dennes

Re: VS Code and Synapse Kernel

Anonymous — Wed, 25 Oct 2023 06:41:29 GMT

Hi @DennesTorres ,
Thanks for using Fabric Community.

While adding the Synapse VS code extension into your Visual Studio, there are some prerequisites which have to be installed in your local desktop. If you install those prerequisites successfully, then extension will auto create the synapse-spark-kernel and install pyspark and other packages. You will able to find this environment after installing all the prerequisites.

Maybe you have installed conda, but forgot to set the environment variable.

When you activate the extension, it will show the initialization progress and output in output channel like this:

Please refer this document for installing the prerequisites : Link1
Hope this helps. Please let us know if you have any further queries.

Re: VS Code and Synapse Kernel

DennesTorres — Wed, 25 Oct 2023 08:27:40 GMT

Hi,

I installed the pre-reqs and variables, but I already suspect what could be.

Either it is because I installed the pre-reqs after the extension (I will need to uninstall the extension and install again) or because VS code has a hard time to update the environment variables unless I reboot. When we have 10 different desktops, each one for different purposes and different stuff opened, reboot is not a good thing.

I will try this and give a feedback later.

Kind Regards,

Dennes

Re: VS Code and Synapse Kernel

Anonymous — Wed, 25 Oct 2023 08:38:40 GMT

Hi @DennesTorres ,

Thanks for the update. Looking forward for your feedback.
Please let us know if you have any further queries.

Re: VS Code and Synapse Kernel

Anonymous — Fri, 27 Oct 2023 09:56:35 GMT

Hi @DennesTorres ,
We haven’t heard from you on the last response and was just checking back to see if your query has been resolved. Please let us know if you have any further queries.

Re: VS Code and Synapse Kernel

DennesTorres — Sat, 28 Oct 2023 13:50:20 GMT

Hi,

All the variables are in place. The requisites are installed.

A restart was done. I uninstalled the extension and installed again.

But nothing worked.

On the VS Code terminal, when opening a notebook and the Synapse window, the following results appear:

The result seems to have no errors, but still the synapse kernel doesn't appear, not allowing the correct notebook execution.

What additional tests could I try ?

Kind Regards,

Dennes

Re: VS Code and Synapse Kernel

Anonymous — Mon, 30 Oct 2023 11:27:49 GMT

Hi @DennesTorres ,
Apologies for the delay in response.
Please follow these trouleshooting steps:

Open the VS Code command palette and select/type 'Python: Select Interpreter'
Choose + Enter interpreter path...
Enter Python interpreter path: path\to\envs\${SYNAPSE_CONDA_ENV_NAME}\python

Please refer this link for further details: VS Code
Hope this helps. Please let us know if you have any further questions.

Re: VS Code and Synapse Kernel

paatrickd — Mon, 30 Oct 2023 14:43:41 GMT

I have a slightly different problem but related to this topic, so I'll post it here.

I did manage to activate the Synapse Kernal in VS Code, however it does not recognize spark. See the code below, spark is of Nonetype.

What should I do?

Re: VS Code and Synapse Kernel

Anonymous — Mon, 30 Oct 2023 14:51:22 GMT

Hi @paatrickd ,
Thanks for using Fabric Community and posting your question.
Can you please create a new post as the initial ask is different from your issue?
We will definitely look into the issue and help.

Thanks for understanding.

Re: VS Code and Synapse Kernel

DennesTorres — Tue, 31 Oct 2023 20:20:35 GMT

Hi,

It was not straightforward, but I think it worked. I will need to make more tests to confirm.

It was difficult to understand the path you asked, but based on the variable, I understand it was something inside the miniconda installation folder.

But there was no python folder there.

Once again, based on the variable, I understand it was related to a folder ENVS, this was present inside the miniconda installation folder, but there was no python folder inside ENVS. There was a folder called synapse-spark-kernel with a python.exe inside it.

I tried this path and received many errors I could not identify. But after closing, opening again and insisting a bit, the synapse-spark-kernel was present in the VS Code.

Thank you!

Kind Regards,

Dennes

Re: VS Code and Synapse Kernel

Anonymous — Wed, 01 Nov 2023 06:04:47 GMT

Hi @DennesTorres ,
Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.

Re: VS Code and Synapse Kernel

Anonymous — Tue, 28 May 2024 13:31:37 GMT

Hi @paatrickd , I am also facing the same issue and stuck at the same error where my fabric-synapse-runtime-1-2 kernel is not recognising spark.

Any help/solution that worked for you will be appreciated.

Thanks.

Re: VS Code and Synapse Kernel

suren_14 — Tue, 03 Sep 2024 17:12:37 GMT

Is there an alternate to conda? Given conda comes at a cost.

Re: VS Code and Synapse Kernel

MRP24 — Sat, 16 Nov 2024 16:51:51 GMT

I had the same issue. after restarting VSCode, I could see the fabric spark kernel

Re: VS Code and Synapse Kernel

DanielBA — Wed, 04 Dec 2024 14:24:07 GMT

Have you found a solution? I am getting the same error...

Re: VS Code and Synapse Kernel

rafasiqueira — Fri, 14 Feb 2025 22:43:12 GMT

Found this on another forum which helped:

"In Fabric or Synapse, the Spark session is automatically created/managed when a cluster is ready for notebook to use. However, if you're running this locally (e.g. VS Code), you need to initiate it manually with SparkSession.builder.getOrCreate().

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("Test").getOrCreate()"