topic Re: Running Notebooks via API with a Specified Session ID in Data Engineering

Running Notebooks via API with a Specified Session ID

rushp — Mon, 21 Apr 2025 10:15:07 GMT

I want to run a Fabric notebook via an API endpoint using a high-concurrency session that I have just manually started.

My approach was to include the sessionID in the request payload and send a POST request, but it ends up creating a run using both the concurrent session and a new standard session.

So, where and how should I include the sessionID in the sample request payload that I found in the official documentation?

I tried adding sessionID and sessionId as a key within "conf" dictionary - it does not work.

POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/items/{{ARTIFACT_ID}}/jobs/instances?jobType=RunNotebook

{
    "executionData": {
        "parameters": {
            "parameterName": {
                "value": "new value",
                "type": "string"
            }
        },
        "configuration": {
            "conf": {
                "spark.conf1": "value"
            },
            "environment": {
                "id": "<environment_id>",
                "name": "<environment_name>"
            },
            "defaultLakehouse": {
                "name": "<lakehouse-name>",
                "id": "<lakehouse-id>",
                "workspaceId": "<(optional) workspace-id-that-contains-the-lakehouse>"
            },
            "useStarterPool": false,
            "useWorkspacePool": "<workspace-pool-name>"
        }
    }
}

IS THIS EVEN POSSIBLE???

Has anyone explored a solution other than using notebookutils.notebook.run()?

I tried but couldn’t find any.
Below are the two jobs that were created after manually initiating a High Concurrency session and sending the API request mentioned above — one that uses the High Concurrency session (which is desired), and another that creates a new independent session (which is not desired).

Re: Running Notebooks via API with a Specified Session ID

Akash_Varuna — Mon, 21 Apr 2025 10:32:19 GMT

Hi @rushp I don’t think it’s currently possible to bind a manually started high-concurrency session directly to a notebook execution via the Fabric API. You might need to explore notebookutils.notebook.run() or raise a support ticket with Microsoft for clarification.

Re: Running Notebooks via API with a Specified Session ID

rushp — Tue, 22 Apr 2025 07:44:32 GMT

Thanks for the clarification, @Akash_Varuna .

Yes, for now, I've implemented a workaround using notebookutils.notebook.run(), and it works.

Re: Running Notebooks via API with a Specified Session ID

Akash_Varuna — Tue, 22 Apr 2025 09:20:01 GMT

@rushp If the solution works please do give a kudos and accept it as a solution so as it will be easier for others to find
Thanks In Advance