Spark Session Staring Issue

[Ryan1304]

Hi All, 

I’ve recently been experiencing significant delays when starting sessions in my notebook environment. Previously, sessions would start in under 30 seconds. However, starting from this week, I’ve noticed the following issues:

- Startup time has increased significantly, now taking at least 3 minutes and 30 seconds, even for a simple or blank notebook.

- Some notebooks take over 10 minutes to start.

- This issue occurs consistently, including with new or empty notebooks.  

For your information: 

Fabric Capacity: F8 SKU, Location: Southeast Asia,

Spark setting: Starter Pool (Node Family: Memory Optimized, Node Size: Medium, Number of Nodes: 1-6) 

Run time version: 1.3 (Spark 3.5, Delta 3.2)

High concurrency: On, Native execution engine: Off

Could anyone please assist in diagnosing and resolving the issue? I’d appreciate any guidance or checks I can perform as well.

Thank you

[BhaveshPatel · Accepted solution]

For faster start up times, Increases the Nodes ( Virtual Machines) and Native Execution Engine ) . If you are using small VMs ( For example F8s, F16..), This problem persists. You should use F64 capacity or more. This is equivalent to Power BI Premium P1 Capacity.

[v-prasare]

We are following up once again regarding your query. Could you please confirm if the issue has been resolved through the support ticket with Microsoft?
If the issue has been resolved, we kindly request you to share the resolution or key insights here to help others in the community. If we don’t hear back, we’ll go ahead and close this thread.
Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.

Thank you for your understanding and participation.

[v-prasare]

Hi @Ryan1304 ,

If your issue still persists, please consider raising a support ticket for further assistance.
To raise a support ticket for Fabric and Power BI, kindly follow the steps outlined in the following guide:

How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

Thanks,

Prashanth Are

MS Fabric community support

[Ryan1304]

Hi @v-prasare 

Thanks for the guidance. I had try Disable High Concurrency mode and also using a dedicated Spark pool with a small size and 3 maximum of nodes. But the time of starting the spark session still remain same. 

[v-prasare]

Hi @Ryan1304,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

[v-prasare]

Hi @Ryan1304,

Can you try below troubleshooting steps and let me know if it helps?

 Starter Pools may take longer to warm up after periods of inactivity. If no nodes are running, startup involves provisioning VMs, which takes time. try switching to a dedicated Spark pool, even a small one, can offer more consistent and faster startup times.

Too many simultaneous sessions starts, pipeline triggers, or data refreshes can congest your capacity. Disable High Concurrency mode temporarily to see if startup improves

navigate to the Admin Portal and selecting Capacity settings, then go to the Monitor section and open Capacity metrics. check key indicators such as CPU and memory pressure, queue length, and overall resource utilization trends. Pay close attention to any signs of throttling or job queuing, which may indicate that your capacity is under strain or experiencing contention, leading to delays in notebook session startup

Thanks,

Prashanth Are

MS Fabric community

If this solution works, please consider marking it as Accept as Solution, this will help other community members easily find answers to similar issues.