Issues running Spark Notebook

[ronan_b6]

Hi I'm runing spark notebooks on Fabric trial, it's been working great and no time out issues, or severe dalys in sessions starting.

But today, notebooks do not start the spark session, returning this type of error:

"Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: System."

I see from the forum that others have experience similar issues, with severe delays in functionality.

Can anyone shed more light on this?

Thanks,

[tobiasny]

Facing the same issue with a F64 Capacity in North Europe.

[abm_chris]

North Europe, now having the same issue, seem to be running ok for me from 10-11am BST

[MMS_007]

Same issue for me. Cannot run pipelines with notebooks (any useful error information), but also cannot run the notebook itself - after around 10 minutes it returns:
SparkCoreError/SessionDidNotEnterIdle: Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: System.

For me it started around 4PM CET yesterday.

In About Power BI section I can see: Your data is stored in North Europe (Ireland) so I guess that's my region (I'm not an admin). It's not a trial but paid Fabric license. 

@Anonymous Ada, what is this issue, what can be done about it, when it will start working?
I cannot develop nor refresh the data in any of workspaces I'm responsible for... It needs to be fixed urgently

[danieletiles]

Facing the same issue, North Europe

[tlanza89_RWL]

I'm having the same issue in the North Central US region as well. The error happens when running a notebook in a pipeline.

"Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Failed to create Livy session for executing notebook"

[daniele_tiles]

In my case is a plain notebook, from scratch. No way to start a spark session

[higgy7]

I am also having the same issue, pipelines with a notebook element fail at the notebook stage with the same error as above.

When i try to start a session within a notebook it tries to start for 10 minutes and then fails to start and disconnects. This issue only started yesterday.

[Anonymous]

Hi all,

We have reported this issue and submitted it to the team.
They have been aware of the issue and the engineers will do their best to resolve it. So please be patient.

Best Regards,

Ada Wang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[jgus]

any updates?

[ToughItOut]

Latest update:

[ToughItOut]

My organization is encountering this issue today as well, also in Canada Central. 
Ive submitted a support request to Unified Support, will update if I learn more.

[ToughItOut]

Looks like its been identified as an issue now, I was in contact with support briefly and passed on some information, then I received a notice around 10PM MST last night. Currently there is a posting in the admin center Service health - Microsoft 365 admin center:

[ToughItOut]

Latest Update:

[ToughItOut]

Final update - I see more posts from other locations, but the notice has changed to resolved. Hoping every is able to start a session now!

• 

   

[Sarien]

It is not fixed yet, just got the error again.

[jonocrotz]

Thanks so much for the updates and visibility.

We are still unable to start up a session as all our pipelines with notebooks are still failing.

I'm waiting on feedback on a P1 ticket from a Partner so will post an update if I hear anything else

[MMS_007]

Which region are you? Maybe they fixed it for Canada. I'm North Europe and I'm still facing the issues... 

[ToughItOut]

Edit - sorry thought you were replying to me. 

Canada Central - I've heard from my team that it seems to be working for them in this region, but clearly this is a larger issue. 

[dmedeiros]

I am also facing the same issue since yesterday both using the trial capacity and a reserved capacity - both from Canada Central

[kausthab]

Facing the same issue in Ontario, Canada

[dver344]

Having a similar issue. It's not timing out, since I don't do a lot of complex processes, but the spark pools usually takes 3 to 5 min to start (before the issue this was 10sec). This all started 24h ago for me (time of writing is 17/07/2024, 2PM Brussels)