topic Notebook Session Not Starting Post Workspace Private Only Configuration in Data Engineering

Notebook Session Not Starting Post Workspace Private Only Configuration

GeetanjaliK — Fri, 23 Jan 2026 08:29:22 GMT

Hi,
We recently changed our Microsoft Fabric workspace setting to “Allow connections only from workspace-level private links.”
After this change, our pipeline started failing only at the Notebook activity, while all other activities such as Lookup, Copy, and ForEach continue to work normally.

The failure message from the Notebook activity is:

Observed Behavior

Lookup, Copy, and ForEach activities still run successfully.
Notebook activity fails immediately (please refer Snip) when trying to start the Spark session.
This issue began only after switching the workspace from public to private-only mode.

Context

Our deployment was done while the workspace was still public, and the pipeline is now running while the workspace is private-only.
Previously, when Spark sessions failed to start, the issue typically resolved automatically within 3–5 hours.
However, after the recent deployment, it has now been over 24 hours, and Spark sessions are still not starting, which is unusual compared to earlier behavior. could you please share your insights on this.

could you please share your insights on this.

Regards,

Geetanjali.

Re: Notebook Session Not Starting Post Workspace Private Only Configuration

deborshi_nag — Fri, 23 Jan 2026 10:58:18 GMT

Hello @GeetanjaliK

In public mode, Spark sessions start on shared starter pools — these are pre‑warmed, Microsoft‑managed clusters available to everyone. But when your workspace blocks public access, Fabric is no longer allowed to use anything on the public network, including those shared pools.

Try to use a custom environment and see if this works.

1. Go to your Fabric workspace > Environment hub

Open Data Engineering / Data Science experience
Select Environment → New Environment

2. Enable custom compute

As workspace admin:
Workspace settings > Data Engineering/Science > Pool tab > “Customize compute configurations for items” = ON
This unlocks custom pool selection inside Environments.

3. Create the environment

Inside the Environment creation panel:

Select a Spark Pool Size:

Fabric provides predefined compute sizes:

Small (4 vCores, 32 GB RAM)
Medium (8 vCores, 64 GB RAM)
Large (16 vCores, 128 GB RAM)
XL (32 vCores, 256 GB)
XXL (64 vCores, 512 GB)
These appear as driver/executor options inside the Environment.
(These options are described in Spark documentation.)

Configure session-level properties:

Number of executors
Executor memory
Driver core count
Executor cores
These settings take effect AFTER Spark starts and remain within Fabric’s pool limits.

4. Save the Environment

5. Open your notebook

6. Attach Environment

From the Compute selector (bottom-left session panel):

Click “Environment”
Select the custom Environment you created

Re: Notebook Session Not Starting Post Workspace Private Only Configuration

v-karpurapud — Tue, 27 Jan 2026 11:41:13 GMT

Hi @GeetanjaliK

Thank you for posting your query in the Microsoft Fabric Community Forum, and thanks to @deborshi_nag for sharing valuable insights.

Could you please confirm if your query has been resolved by the provided solution?If you have any more questions, please let us know and we’ll be happy to help.

Regards,

Microsoft Fabric Community Support Team

Re: Notebook Session Not Starting Post Workspace Private Only Configuration

stoic-harsh — Tue, 27 Jan 2026 19:09:48 GMT

Hi @GeetanjaliK,

Could you try temporarily switching the workspace back to public access, trigger the Notebook once to let Spark initialize, then re-enable private-only access.

Please do share whatever works for you, so that it helps us all in such scenarios.

Re: Notebook Session Not Starting Post Workspace Private Only Configuration

v-karpurapud — Thu, 29 Jan 2026 04:58:52 GMT

Hi @GeetanjaliK

We have not received a response from you regarding the query and were following up to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.

Thank You.