Forum Discussion
Notebook Session Not Starting Post Workspace Private Only Configuration
Hello GeetanjaliK
In public mode, Spark sessions start on shared starter pools — these are pre‑warmed, Microsoft‑managed clusters available to everyone. But when your workspace blocks public access, Fabric is no longer allowed to use anything on the public network, including those shared pools.
Try to use a custom environment and see if this works.
1. Go to your Fabric workspace > Environment hub
- Open Data Engineering / Data Science experience
- Select Environment → New Environment
2. Enable custom compute
As workspace admin:
Workspace settings > Data Engineering/Science > Pool tab > “Customize compute configurations for items” = ON
This unlocks custom pool selection inside Environments.3. Create the environment
Inside the Environment creation panel:
Select a Spark Pool Size:
Fabric provides predefined compute sizes:
- Small (4 vCores, 32 GB RAM)
- Medium (8 vCores, 64 GB RAM)
- Large (16 vCores, 128 GB RAM)
- XL (32 vCores, 256 GB)
- XXL (64 vCores, 512 GB)
These appear as driver/executor options inside the Environment.
(These options are described in Spark documentation.)
Configure session-level properties:
- Number of executors
- Executor memory
- Driver core count
- Executor cores
These settings take effect AFTER Spark starts and remain within Fabric’s pool limits.
4. Save the Environment
5. Open your notebook
6. Attach Environment
From the Compute selector (bottom-left session panel):
- Click “Environment”
- Select the custom Environment you created
Hello GeetanjaliK
In public mode, Spark sessions start on shared starter pools — these are pre‑warmed, Microsoft‑managed clusters available to everyone. But when your workspace blocks public access, Fabric is no longer allowed to use anything on the public network, including those shared pools.
Try to use a custom environment and see if this works.
1. Go to your Fabric workspace > Environment hub
- Open Data Engineering / Data Science experience
- Select Environment → New Environment
2. Enable custom compute
As workspace admin:
Workspace settings > Data Engineering/Science > Pool tab > “Customize compute configurations for items” = ON
This unlocks custom pool selection inside Environments.
3. Create the environment
Inside the Environment creation panel:
Select a Spark Pool Size:
Fabric provides predefined compute sizes:
- Small (4 vCores, 32 GB RAM)
- Medium (8 vCores, 64 GB RAM)
- Large (16 vCores, 128 GB RAM)
- XL (32 vCores, 256 GB)
- XXL (64 vCores, 512 GB)
These appear as driver/executor options inside the Environment.
(These options are described in Spark documentation.)
Configure session-level properties:
- Number of executors
- Executor memory
- Driver core count
- Executor cores
These settings take effect AFTER Spark starts and remain within Fabric’s pool limits.
4. Save the Environment
5. Open your notebook
6. Attach Environment
From the Compute selector (bottom-left session panel):
- Click “Environment”
- Select the custom Environment you created