Forum Discussion

DennesTorres's avatar
DennesTorres
Impactful Individual
2 years ago

Instable enviroment after setting environment objects

Hi,   I had to create an Environment object and set it to the workspaces I'm working on.   The environment is on default configurations except by a spark property I enabled (dynamic partitions). ...
data engineering
willmasse's avatar
willmasse
Frequent Visitor
2 years ago

I'm getting these same exact errors too. Started in the past few days and they happen very frequently. Also using a enviorment but not sure if that was the issue. I had notebooks running fine until the past few days. 

  • DennesTorres's avatar
    DennesTorres
    Impactful Individual
    2 years ago

    Hi, Will,

     

    It's interesting. The fact you are getting the same errors and you are also using environments points to a pattern. The Environment objects seems to create problems to the start of spark sessions, although we have no further evidence and we can't reproduce, it seems a pattern.

    Kind Regards,

    Dennes

    • willmasse's avatar
      willmasse
      Frequent Visitor
      2 years ago

      Checking in, I haven't gotten these errors in the past few days and my pipelines with notebooks are seemingly running fine now. Wondering if the issue has gone away for you to? 

       

      I'm assuming this is entirely something MS server side they must have quietly addressed? 

      • DennesTorres's avatar
        DennesTorres
        Impactful Individual
        2 years ago

        Hi,

        I just got one:

        SparkCoreError/SessionDidNotEnterIdle: Livy session has failed. Error code: SparkCoreError/SessionDidNotEnterIdle. SessionInfo.State from SparkCore is Error: Session did not enter idle state after 10 minutes. Source: SparkCoreService. Referenced notebook stacktrace: at Notebook NB - Load Configuration, cell 2

         

        Kind Regards,

         

        Dennes