Hi everyone,
I am new here and to the world of Microsoft Fabric. I am trying to run a notebook in VS Code (Desktop) on macOS (M3), but I am unable to run the notebook using the Fabric instance (Spark). While I can load, open, update, and retrieve all notebooks from my workspace, I can't run cells with code like:
df = spark.sql('SELECT * FROM my_lakehouse.my_table')
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[3], line 1
----> 1 spark.sql('SELECT * FROM my_lakehouse.my_table')
AttributeError: 'NoneType' object has no attribute 'sql'
I’ve noticed that many users have had similar issues and I’ve tried all the proposed fixes, but nothing works. I have reinstalled everything multiple times, ensuring all requirements (conda, java, etc.) are met. I have also experimented with both environments (1-1 & 1-2). Yet, I still get the following error (log):
root ERROR Failed to initialize Spark Lighter variables. An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
: java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
Has anyone encountered this issue before? I’d really appreciate any help or suggestions.
Thanks!
