topic Can't access the Lakehouse when using notebook in pipeline in Pipelines

Can't access the Lakehouse when using notebook in pipeline

Anushka96 — Fri, 30 Aug 2024 20:11:42 GMT

When using notebooks with pipeline I was encountered with following error.

"Py4JJavaError","evalue":"An error occurred while calling o4717.table.\n: java.lang.RuntimeException: Request failed: HTTP/1.1 403 Forbidden\n\tat com.microsoft.fabric.spark.metadata.Helpers$.executeRequest

It seems like a permission issue I have faced it when script running using notebooks. I was able to avoid it by draging the lakehouse to cell for once. In notebooks I am using absoulte path for tables. Anyone have a soloution for this?

Re: Can't access the Lakehouse when using notebook in pipeline

Anonymous — Mon, 02 Sep 2024 01:36:44 GMT

Hi Anushka96 ,

This error usually means that you have been denied permissions when trying to access a resource.

I think you can do something below:

1. make sure you are using the correct paths in your code. For example, use an absolute path instead of a relative path to access a table in Lakehouse.

2. Make sure you have the proper permissions to access that resource. If it's an API call, confirm that your API key or token is valid and that you have sufficient permissions.

3. Confirm that the URL you are requesting is correct, sometimes misspellings or wrong paths can cause 403 errors.

4. Of course sometimes firewalls or security settings can block the request, make sure these settings allow your request to go through.

You can also check out this post below to gain more knowledge on the document: Troubleshoot security and access control issues - Azure Data Factory & Azure Synapse | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: Can't access the Lakehouse when using notebook in pipeline

Anushka96 — Mon, 02 Sep 2024 02:26:52 GMT

Notebook even tells that error when I am running them individually as well without using a pipeline. I am using abfs paths and also it working sometime.I am a workspace administrator so this cannot due to permission errors.

Re: Can't access the Lakehouse when using notebook in pipeline

ML15 — Wed, 04 Sep 2024 06:42:23 GMT

I got the same error, but this fixed it for me:
https://community.fabric.microsoft.com/t5/Data-Engineering/An-error-occurred-while-calling-o321-sql/m-p/4130849#M3870