Mounting NB on- the-fly

Question

I am running some codes in a standalone notebook that is currently giving me the following error
Spark SQL queries are only possible in the context of a lakehouse. Please attach a lakehouse to proceed
&nbsp;
How can I progrmaatically mount/add a lakehouse to a NB prior to execution of that code. I tried following 
notebookutils.fs.mount('abfss://ws_id@onelake.dfs.fabric.microsoft.com/lh_id','/lakehouse/default')
and then I tried running the spark code which still gave me the same error as above.
&nbsp;
Is it doable?
&nbsp;

frithjof_v · Answer

I see, I&nbsp;will do a test on my side if I find the time to do the test.&nbsp;Btw I see you are using notebookutils whereas the blog post is using mssparkutils.&nbsp;Will it work if you use mssparkutils instead of notebookutils?

frithjof_v · Answer

Here is a solution which doesn't involve mounting or attaching Lakehouses to the Notebook:&nbsp;Perhaps you can use temporary views to be able to work with Spark SQL without having a default Lakehouse for your Notebook.&nbsp;In this example I have a Notebook without any default Lakehouse and without any mounted/attached lakehouses.&nbsp;I create a dataframe to connect to data from a lakehouse using the abfss path, and create a temporary view based on that dataframe.&nbsp;&nbsp;&nbsp;df = spark.read.load("abfss://&lt;workspaceID&gt;@onelake.dfs.fabric.microsoft.com/&lt;lakehouseID&gt;/Tables/&lt;tableName&gt;")

df.createOrReplaceTempView("MyView_Read")&nbsp;&nbsp;I use Spark SQL to do some modifications to the temporary view, and save it as another temporary view.&nbsp;%%sql
CREATE OR REPLACE TEMPORARY VIEW MyView_Write AS SELECT *, Current_Timestamp as LoadTime FROM MyView_Read;&nbsp;&nbsp;I create a new dataframe from the new temporary new.I then write this new dataframe to another lakehouse table in another workspace.&nbsp;df_write = spark.sql("SELECT * FROM MyView_Write")

df_write.write.mode("append").save("&lt;anotherWorkspaceID&gt;@onelake.dfs.fabric.microsoft.com/&lt;anotherLakehouseName&gt;/Tables/&lt;anotherTableName&gt;")​&nbsp;&nbsp;This seems to work fine for me.

frithjof_v · Answer

Hopefully something in this blog post can help achieving what you request: https://fabric.guru/how-to-mount-a-lakehouse-and-identify-the-mounted-lakehouse-in-fabric-notebook

smpa01 · Answer

This post describes how to mount a NB which is exactly similar to what I have provided (good so far)
but it does not execute a code in a Notebook manually attached to a Lakehouse would have successfully executed otherwise.
&nbsp;
The objective of progrmatic mounting is to see if the succedding code executes correctly as it would have in a manually attached lakehouse.

smpa01 · Answer

frithjof_v&nbsp; i will check and let you know.

Forum Discussion

Mounting NB on- the-fly

10 Replies

Related content

Field Parameter Rank On The Fly

Earlier function on the fly

Notebook - mount a lakehouse fails

Calculating Changes on the fly between two tables

Fabric could not load ADF pipeline after mounting