topic Re: Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession in Data Engineering

Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession

Anonymous — Thu, 05 Oct 2023 07:54:33 GMT

Hi I'm runing spark notebooks on Fabric trial since last month and it's been working great.

But just from today afternoon, notebooks do not start the spark session, returning this type of error:

Livy session has failed. Error code: SparkCoreError/UnexpectedSessionState. SessionInfo.State from SparkCore is Error: Encountered an unexpected session state Error while waiting for session to become Idle. Error description:"Cluster creation failed due to an unexpected system error from a runtime related dependency.

Can any expert help ASAP?

Thanks,

Re: Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession

Anonymous — Thu, 05 Oct 2023 12:45:20 GMT

Hi Anonymous - Thanks for using Fabric Community.

I would like to check are you still facing this issue?
Can you please some more details like in which region you are facing this issue and along with attach screenshots of the issue?

Re: Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession

Anonymous — Thu, 05 Oct 2023 20:45:27 GMT

Hi Anonymous,

Thanks for reaching out.

The issue somehow no longer appears, but I would like to share the details for your benefit.

Where Fabric client is accessed, i.e. where web-browser connected from: Auckland, New Zeland
Azure region of Power BI/Fabric: Australia East.
Trial Capacity region: Southeast Asia
Issue first detected from (all in NZ local time):
- From 5 Oct 2023, 4:12 pm
- Happened till 5 Oct 2023 8:40pm
Here's the screenshot of an error (though there's not much info):

Hope the information helps to identify the cause.

Regards,

Re: Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession

Anonymous — Fri, 06 Oct 2023 04:23:42 GMT

Hi Anonymous ,

Thanks for your inputs, We will look into it.

Glad to know that your query got resolved.

Please continue using fabric for help regarding your issues.

Re: Unable to start spark session: Livy session has failed. Error code: SparkCoreError/UnexpectedSession

FelixZent — Sun, 21 Jul 2024 20:43:32 GMT

Is there any Solution for this problem?
I get the exact same error and then it disappears bur reappears after some time.