Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
FilipO
Advocate I
Advocate I

Slow starter pool session initialization

I am currently experimenting with a F2 capacity. Whenever I try to run a notebook it takes somewhere between 2 and 2.5 minutes to start the spark cluster even though I'm using the starter pool. All settings related to Spark are left as default for the workspace. Tried both running things manually as well as triggering the notebook from data factory.

 

FilipO_0-1698763546062.png

 

 

If I go into a trial workspace and run a notebook the spark session is started in just a few seconds. This makes me think that it might be a limitation depending on which capacity you are using? But the documentation gives no indication of this: https://learn.microsoft.com/en-us/fabric/data-engineering/spark-compute

23 REPLIES 23

I spoke with support yesterday and they claimed that 3 to 4 minutes to start running a notebook is normal... And then when I was demonstrating the start of a notebook it simply failed... 😀 

Hey there. I'm still having this issue with slow start ups. I notice that this problem doesn't occur when starting a notebook with the starter pool (which fires up within seconds). So if the starter pool can start up that quickly, it doesn't make sense for a custom pool to take much longer, yeah? Imagine the same notebook being used in parallel and each of them taking that many minutes and times that by the number of jobs in the queue waiting to go through the notebook start up wait time... 

Yeah, it is furstrating that they can take several minutes to start up.

 

One option is to use runMultiple to execute your notebooks. You can define the dependencies and then execute all of the notebooks on a single compute. It only has to start up once rather than for each notebook.

 

Using runMultiple To Orchastrate Notebook Execution in Microsoft Fabric

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.