topic Cannot Start Spark Session in Data Engineering

Cannot Start Spark Session

jbanas — Wed, 28 Feb 2024 17:50:36 GMT

Running into the following error code when trying to start a Spark Session.

"Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: System

Everything was working great until today. I see some others had some similar errors, that might have been related to Custom Environments. I am running the default environment. Any ideas?

Re: Cannot Start Spark Session

iceenine — Wed, 28 Feb 2024 20:03:44 GMT

I am receiving the same thing in East US and East US 2 for the past several hours. We have created new Synapse instances and Spark pools with default environments to validate and see the same result.

There are several questions on Q&A that MS support has been closing suggesting that users come here for support instead so hopefully we see some engagement soon.

Re: Cannot Start Spark Session

iceenine — Wed, 28 Feb 2024 20:58:22 GMT

Following up, the product group acknolwedged an issue about 30 mins ago in another thread but no ETA on a fix yet.

Re: Cannot Start Spark Session

Anonymous — Thu, 29 Feb 2024 07:29:26 GMT

Hi @jbanas ,

Thanks for using Fabric Community.
I would like to check are you still facing this issue?
Can you please share the region you are observing this issue?

Re: Cannot Start Spark Session

jbanas — Thu, 29 Feb 2024 14:52:54 GMT

Hello,

East US 2.

The issue was sporadic throughout yesterday, with a few hours of complete downtime. However, today it seems to be functioning properly.

Re: Cannot Start Spark Session

Anonymous — Fri, 01 Mar 2024 07:10:23 GMT

Hi @jbanas ,

Glad to know that issue got resolved. Please continue using Fabric Community for your further queries.