topic Re: Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric in Data Engineering

Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric

antonioman — Wed, 02 Oct 2024 17:04:22 GMT

Can someone explain the steps I need to follow to call a Python script stored in Azure Blob Storage from a Spark Job Definition in Microsoft Fabric? When I provide the ADLS-Gen2 Path URL and then run the job, I am getting a 403 access error for the Python file.

Re: Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric

Anonymous — Thu, 03 Oct 2024 02:33:03 GMT

Hi @antonioman ,

403 error means you not have premissions to access these resources. Can you please share some more detail information about this your opeartions? They should help us clarify your scenario and test.

In addition, what type of capacity are you worked? Trial or fabric capacity?

The trial capacity is hosted in the virtual environment, so you may face some issue to invoke resources across trial and payment environment.

Regards,

Xiaoxin Sheng

Re: Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric

antonioman — Thu, 03 Oct 2024 06:34:10 GMT

I am currently using an F2 capability. My scenario is as follows. I have created a Spark Job Definition and I have hosted Python code in an Azure Blob Storage. This Azure Blob Storage is in the same resource group as Fabric. The code to be used in the Spark Job Definition is the one hosted in the Storage. When I launch the execution, I get a 403 error when trying to access the code files. I understand that I need to grant Fabric permissions to access the resource, but I don’t see any documentation on the website

Re: Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric

Anonymous — Tue, 29 Oct 2024 07:14:21 GMT

HI @antonioman,

Can you please share the same code that you used? I will tried on it and share the insight here if I got them from testing.

Regards,

Xiaoxin Sheng

Re: Error 403 Accessing Python Script in Azure Blob Storage from Spark Job in Microsoft Fabric

richbenmintz — Wed, 30 Oct 2024 12:45:34 GMT

try creating a connection to the Blob Storage account, see the doc at the link below. use blob as the connection type

https://learn.microsoft.com/en-us/fabric/data-factory/connector-azure-data-lake-storage-gen2