Spark Session Staring Issue

[Ryan1304]

Hi All, 

I’ve recently been experiencing significant delays when starting sessions in my notebook environment. Previously, sessions would start in under 30 seconds. However, starting from this week, I’ve noticed the following issues:

- Startup time has increased significantly, now taking at least 3 minutes and 30 seconds, even for a simple or blank notebook.

- Some notebooks take over 10 minutes to start.

- This issue occurs consistently, including with new or empty notebooks.  

For your information: 

Fabric Capacity: F8 SKU, Location: Southeast Asia,

Spark setting: Starter Pool (Node Family: Memory Optimized, Node Size: Medium, Number of Nodes: 1-6) 

Run time version: 1.3 (Spark 3.5, Delta 3.2)

High concurrency: On, Native execution engine: Off

Could anyone please assist in diagnosing and resolving the issue? I’d appreciate any guidance or checks I can perform as well.

Thank you

[v-prasare]

Hi @Ryan1304,

Can you try below troubleshooting steps and let me know if it helps?

 Starter Pools may take longer to warm up after periods of inactivity. If no nodes are running, startup involves provisioning VMs, which takes time. try switching to a dedicated Spark pool, even a small one, can offer more consistent and faster startup times.

Too many simultaneous sessions starts, pipeline triggers, or data refreshes can congest your capacity. Disable High Concurrency mode temporarily to see if startup improves

navigate to the Admin Portal and selecting Capacity settings, then go to the Monitor section and open Capacity metrics. check key indicators such as CPU and memory pressure, queue length, and overall resource utilization trends. Pay close attention to any signs of throttling or job queuing, which may indicate that your capacity is under strain or experiencing contention, leading to delays in notebook session startup

Thanks,

Prashanth Are

MS Fabric community

If this solution works, please consider marking it as Accept as Solution, this will help other community members easily find answers to similar issues.