topic Re: Error when calling Notebook from Rest API in Data Engineering

Error when calling Notebook from Rest API

qnguyen12 — Thu, 18 Dec 2025 23:50:48 GMT

I created a Notebook and ran it manually without issues. The Notebook executes an AI function (ai.classify), accepts input parameters, and then inserts the results into the Lakehouse.

However, I encounter a problem when using the REST API. I created a Service Principal and added the scopes as instructed on the website (Job Scheduler - Run On Demand Item Job - REST API (Core) | Microsoft Learn).

In View Recent Runs, the status shows Success.

But when I check the run details, I see the following error: Spark_System_MetaStore_UnableToFetchMWCToken (Error inserting data using the Metastore. The Metastore keeps track of tables and views used by Spark. This error typically means that your code referred to a table or view that did not exist, or a column within a table or view that does not exist. 1. Ensure the table or view you are referring to exists. 2. Ensure all columns that that are being referenced in the insert exist in the target tables. 3. Check the logs for this Spark application. Inspect the logs for a clearer indication of which table relation is causing this issue.)

If I use the bearer token of my account (not service principal), the run is good.

I’m wondering which scope or additional step I need to configure to ensure the run completes successfully.

Re: Error when calling Notebook from Rest API

qnguyen12 — Fri, 19 Dec 2025 01:17:24 GMT

I added permission to lakehouse for the service princial but got new errors

Re: Error when calling Notebook from Rest API

tayloramy — Fri, 19 Dec 2025 15:00:35 GMT

Hi @qnguyen12,

It looks like you still need to add permissions to the service principal on the ADLS Gen 2 storage in Azure.

Are you using an external ADLS Gen 2 storage area outside of Fabric?

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

Re: Error when calling Notebook from Rest API

qnguyen12 — Fri, 19 Dec 2025 21:06:30 GMT

Hi @tayloramy , I don't use ADLS Gen 2 storage area outside of Fabric.
I tried to add Azure Data Lake permission but failed because I don't use the service in Azure?

Re: Error when calling Notebook from Rest API

qnguyen12 — Sat, 20 Dec 2025 22:46:11 GMT

I add the service principal as Contributor in "Manage access" and it's good now.