Forum Discussion

Shawn_Eary's avatar
Shawn_Eary
Advocate V
2 years ago

Databricks ConnectV2 for Microsoft Fabric?

Databricks ConnectV2 lets you use Visual Studio Code as an IDE to debug Databricks notebooks: Debug your code and notebooks by using Visual Studio Code | Databricks Blog Does Microsoft Fabric or ...
data engineering
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi Shawn_Eary ,
Following up to check whether your issue has been resolved. Please let us know if you have any further queries.

Shawn_Eary's avatar
Shawn_Eary
Advocate V
2 years ago

The extension was and still is installed, but now I'm getting a different error and I'm not sure why. Now this line:

 

spark.read.format("csv").option("header", "true").load(f)

 

spits out the error:
NameError: name 'spark' is not defined
 
When I started up the debugger, I was given an option to choose between the Python Environments and Existing Jupyter Server:

I chose Environments -> Python 3.11.6:

 Because I didn't know of a Jupyter Server URL that MS Fabric provides.

Is there some way for me to get access to a Spark context and the files in my MS Fabric instance instead of apparently running my Python Notebook locally?

My understanding is that Databricks already has the functionality I'm looking for.

 

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Shawn_Eary ,
    The issue which you are facing is due to the Python environment which you are using. You should use the synapse-spark-kernel for your notebook code to run without any error.
    I have attached the screenshot for your reference.

     


    While adding the Synapse VS code extension into your Visual Studio, there are some prerequisites which have to be installed in your local desktop. If you install those prerequisites successfully, then extension will auto create the synapse-spark-kernel and install pyspark and other packages. You will able to find this environment after installing all the prerequisites.

     

    Please refer this document for installing the prerequisites : Link 

     

    Yes, you can connect to OneLake via Azure Databricks.
    Please refer this link for connecting to Databricks: Link2 

    Hope this helps. Please do let us know if you have any further questions.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Shawn_Eary ,
      Following up to check whether your issue has been resolved. Please let me know if you have any further questions.