Forum Discussion
Instable enviroment after setting environment objects
Hi,
I had to create an Environment object and set it to the workspaces I'm working on.
The environment is on default configurations except by a spark property I enabled (dynamic partitions).
However, after I set the environment on the workspaces, I'm getting an error message more often:
"SparkCoreError/SessionDidNotEnterIdle: Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: SparkCoreService."
I'm not sure what this means, this usually happens when I'm starting a new execution.
I'm not sure if I should take it to support or how, because I don't know how to reproduce, it's just an error message I'm getting way more often after setting the environment on the workspaces.
Kind Regards,
Dennes
12 Replies
Hi DennesTorres ,
Thanks for using Fabric Community.
I would be able to guide you if you can explain your issue with few more details.
I am unable to understand the issue based on your information provided.
Can you please share the documentation link you are following, so I can get better idea over your issue?Hi,
The issue:
Recently I'm receiving error messages when starting new spark sessions, running notebook code blocks. The errors are way more frequent than before.
The error message:
"SparkCoreError/SessionDidNotEnterIdle: Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: SparkCoreService."
Sometimes I don't receive the error, but the session takes many minutes to start.
I can't reproduce, it's just something which is happening more often.
What was changed to cause this?
I created an Environment object using the default options. The only customization on the environment object was one custom spark configuration for dynamic partition. (I doubt the kind of configuration may cause this).
The environment was linked to the workspace on the workspace settings.
After this change, the errors when starting Spark sessions became way more usual.
Kind Regards,
DennesI have the same issue, any updates?
I'm getting these same exact errors too. Started in the past few days and they happen very frequently. Also using a enviorment but not sure if that was the issue. I had notebooks running fine until the past few days.
Hi, Will,
It's interesting. The fact you are getting the same errors and you are also using environments points to a pattern. The Environment objects seems to create problems to the start of spark sessions, although we have no further evidence and we can't reproduce, it seems a pattern.
Kind Regards,
DennesChecking in, I haven't gotten these errors in the past few days and my pipelines with notebooks are seemingly running fine now. Wondering if the issue has gone away for you to?
I'm assuming this is entirely something MS server side they must have quietly addressed?
