topic Re: Microsoft Fabric Notebook data from API Error in Pipelines

Microsoft Fabric Notebook data from API Error

Drei — Sat, 17 Feb 2024 14:12:51 GMT

Failed to upload blockAuthentication Failed with Access token validation failed

This is the error i always get when I try to import data from an API. It is a decent amount of data 36000 json files. I have asked help to any chat ai bot available but cannot solve it.

Sidenote: It is possible that I get limited on the API, but every copilot and friends told me it is related to Azure. But I did every troubleshooting step possible. (If it is related to Statsbomb API call , it shouldnt be, as we are allowed 50.000 cals/min and my code ran 35 mins for 3000 json files.)

I will copy the full error underneath.
My Fabric = a code in spark notebook to import data from an API (Statsbomb).
Trouble shooting = I have linked my gen2 account to the workspace. I have upgraded to paid verson F2 instead of trial. But nothing seems to help.

Full error code:

Re: Microsoft Fabric Notebook data from API Error

v-cboorla-msft — Mon, 19 Feb 2024 07:12:47 GMT

Hi Drei

Thanks for using Microsoft Fabric Community.

Apologies for the inconvenience that you are facing here.

The “Access token validation failed” error typically occurs when there’s an issue with the authentication process. Here are some troubleshooting steps that might help you.

Check Your Authentication Credentials: Ensure that you have the correct access token or API key for accessing the StatsBomb API. Double-check that you’re using the right token in your Notebook.
Verify Token Scope and Permissions: Make sure your token has the necessary permissions to access the specific API endpoint. Some APIs require specific scopes (e.g., read, write, or specific data access). Confirm that your token has the required scope.
Token Expiry: Access tokens have a limited lifespan. Verify that your token is still valid. If it has expired, obtain a fresh token.
Rate Limiting: Some APIs have rate limits. Make sure you’re not exceeding the allowed number of requests per minute or hour.
Debugging with Postman: Use tools like Postman to manually test the API request. Capture a valid access token and simulate the API call outside your Notebook. Observe any error messages or issues.

I hope this information helps. Please do let us know in case if you have further questions.

Thanks.

Re: Microsoft Fabric Notebook data from API Error

HimanshuS-msft — Tue, 20 Feb 2024 18:19:14 GMT

Hello Drei ,
From the error , its tough to tell which system is throwing the exception . if i were you i could have done this .
1. Narrow down to a smaller data set . may be just one JSON file and see is that works .
2.Tried to run the the API call using curl and Postman and see how it behaves .
Thanks
Himanshu

Re: Microsoft Fabric Notebook data from API Error

v-cboorla-msft — Wed, 21 Feb 2024 09:49:49 GMT

Hi Drei

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thanks.

Re: Microsoft Fabric Notebook data from API Error

Drei — Wed, 21 Feb 2024 10:01:25 GMT

Hi all! I am very sorry to respond this late. But I have remove the workspace. It was kind of urgent and it did not work for me. No solution helped as I cannot lower the amount of data i need to import. I did try in partitions but did not help. I encounter a lot of errors in fabric when moving large amounts of data and will look for other solutions. I have been able to get all the data via local python code on a drive. So it is not an API problem but the problem is with the amount of data fabric can handle at once. Another example is when I have 30.000 rows of data and I standardize _z the data, I have it in my data frame, but when I write to parquet 70% of the times it fails.

Sorry to have bothered you all and I am glad to see how helpful you are! I hope fabric improves and become the main tool for data. As it is in concept a great idea!

Re: Microsoft Fabric Notebook data from API Error

MrNeo — Thu, 25 Apr 2024 05:41:54 GMT

I got the same error when tried to convert an excel file to a parquet one and store it in the data lakehouse. Surprisingly, resetting the standard session to the concurrent one and back to the standard helped. But maybe I just needed to reset the session, anyway, the error went away after that.