topic Re: Notebook - mount a lakehouse fails in Data Engineering

Notebook - mount a lakehouse fails

frithjof_v — Sun, 18 Aug 2024 12:09:49 GMT

I'm trying to mount a lakehouse from another workspace to my Notebook by using this code:

notebookutils.fs.mount( "abfss://<workspace_id>@msit-onelake.dfs.fabric.microsoft.com/<lakehouse_id>", "/test" )

NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

(Yes, I am replacing the <workspace_id> and <lakehouse_id> with the GUID's of my workspace and lakehouse)

I get an error:

Py4JJavaError: An error occurred while calling z:notebookutils.fs.mount. : java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]

Re: Notebook - mount a lakehouse fails

Anonymous — Mon, 19 Aug 2024 03:02:08 GMT

Hi frithjof_v ,

Use this command to mount lakehouse to the current notebook.

mssparkutils.fs.mount(“abfss://<workspace_id>@onelake.dfs.fabric.microsoft.com/<lakehouse_id>”, “/lakehouse/test”) #mountPoint such as '/lakehouse/default'

The successful mount is shown below:

You can also make sure that the mount is successful by executing this command below to list all the mount points.

mssparkutils.fs.mounts()

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Notebook - mount a lakehouse fails

frithjof_v — Mon, 19 Aug 2024 18:36:51 GMT

Thanks, it worked!

however MS recommends to upgrade to notebookutils instead of mssparkutils:

NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

I get an error when trying to do the mount using Notebookutils:

NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

Does this code work on your side?

Re: Notebook - mount a lakehouse fails

Anonymous — Tue, 20 Aug 2024 07:08:16 GMT

Hi frithjof_v ,

I failed at first with this code because the default timeout may not be enough, after increasing the timeout I succeeded:

notebookutils.fs.mount ( “abfss://<workspace_id>@msit-onelake.dfs.fabric.microsoft.com/<lakehouse_id>”, “/test”. {“timeout”: 600} )

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

Re: Notebook - mount a lakehouse fails

frithjof_v — Tue, 20 Aug 2024 07:41:58 GMT

Thanks!

It seems perhaps the default timeout of this function is too low, perhaps that it something which the dev team should adjust.

Thanks for showing the workaround!

Re: Notebook - mount a lakehouse fails

Anonymous — Tue, 20 Aug 2024 08:25:10 GMT

Hi frithjof_v ,

Could you please mark this helpful post as “Answered”?

This will help others in the community to easily find a solution if they are experiencing the same problem as you.

Thank you for your cooperation!

Best Regards,
Yang
Community Support Team

Re: Notebook - mount a lakehouse fails

dbeavon3 — Thu, 23 Jan 2025 21:30:03 GMT

Hi Anonymous

These failures are happening in our default LH mounts, even before our notebook cells start executing. Is this the same bug?

Have you reported this bug to the PG yet? Is there an ICM number that you can share? If you cannot share that identifier here, can you please add the details to the internal notes on my SR:
TrackingID#2501220040016849

This needs to be added to the "known issues" list, if customers are expected to use workarounds to avoid these bugs in their default mounts.

Customers should not be forced to make sure that the (default) mount is successful by executing this command to list all the mount points. mssparkutils.fs.mounts().

... It would be far better if the PG would crash the notebook right at the start, as opposed to allowing the notebook to begin execution with an invalid session/environment. No Fabric customer wants to deal with an invalid session; and we don't want to experience random and unpredictable errors in our code, as a result of a software bug from Microsoft. The experience is very confusing to a pyspark-notebook-developer, and is even more confusing when a developer must explain to their users why their workloads are dying. These end-users are NOT likely to believe us (an average pyspark-notebook-developer) when we blame the Microsoft Fabric platform for our unreliable workloads. Hopefully this makes sense. (... yet it has been months since you shared your workaround and there is still no relief from the PG, and nothing appears in the "known issues" list.)