topic Re: Mounting Azure Blob Storage Container using Fabric Notebook (PySpark) in Data Engineering

Mounting Azure Blob Storage Container using Fabric Notebook (PySpark)

smoqt — Wed, 28 Feb 2024 20:48:21 GMT

I'm attempting to mount an Azure Blob Storage Container within a Fabric Notebook as suggested here.

I am setup as a Storage Blob Data Contributor on the container (which I have seen suggested).

The code:

from notebookutils import mssparkutils accountKey = mssparkutils.credentials.getSecret("<vaultURI>", "<secretName>") mssparkutils.fs.mount( "abfss://<mycontainer>@<accountname>.dfs.core.windows.net", "/test", {"accountKey":accountKey} )

I get an error: "Mount failed with operationId <operationId>, get detail from /var/log/blobfuse/blobfusev2.log"

When checking the log, I notice the following sequence:

|2024-02-28T17:44:15.982905+00:00 vm-65b06715 blobfusev2[2878]: Testing authetication to /<some-guid>/test/ ...

|2024-02-28T17:44:16.036071+00:00 vm-65b06715 blobfusev2[2878]: Unknown failure: [json.exception.type_error.302] type must be string, but is number

|2024-02-28T17:44:16.036174+00:00 vm-65b06715 blobfusev2[2878]: Failed to connect to the storage container. There might be something wrong about the storage config, please double check the storage account name, account key/sas token/OAuth access token and container name. errno=1600

Naturally, I have double-checked the accountname and key as well as tried other methods like using a sas token. So far, I'm have no success mounting anything (I also tried mounting a lakehouse but it just times out with no errors in the log). Am I missing something?

Re: Mounting Azure Blob Storage Container using Fabric Notebook (PySpark)

smoqt — Thu, 29 Feb 2024 00:00:19 GMT

As a test, I created two new containers within the same storage account and was able to mount both. I've tried mounting other, pre-existing containers but none have mounted successfully.

Both were empty containers. I then unmounted one of the containers, added an empty txt file to the container, tried mounting it again but got the error.

Next, I deleted the file, tried once more and it mounted successfully.

Re: Mounting Azure Blob Storage Container using Fabric Notebook (PySpark)

Anonymous — Thu, 29 Feb 2024 05:31:11 GMT

Hi @smoqt
Thanks for using Fabric Community.
Can you please provide the livy id/driver log or cluster id /session id for the above error?

Re: Mounting Azure Blob Storage Container using Fabric Notebook (PySpark)

Anonymous — Fri, 01 Mar 2024 18:14:53 GMT

Hi @egons11

We haven’t heard from you on the last response and was just checking back to see if you can provide the details asked above. We will definitely try to help.
Thanks

Re: Mounting Azure Blob Storage Container using Fabric Notebook (PySpark)

raaph — Fri, 19 Jul 2024 12:08:02 GMT

I have the same issue: In a notebook in Synapse I can not mount a container that has files inside, but it works if the container is empty. I mount it with 'linkedService' and not 'accountKey' like smoqt.