Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Slow starter pool session initialization

I am currently experimenting with a F2 capacity. Whenever I try to run a notebook it takes somewhere between 2 and 2.5 minutes to start the spark cluster even though I'm using the starter pool. All s...
admin
data engineering
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi Anonymous ,

Apologies for the issue you have been facing. We have an update from the team.
 We got impacted by this limitation on supporting 1 node starter pools mainly for F2 capacity sku. Team have added this to Known Issues section in the document and the latest version should be published end of this week. We are working on a fix to enable 5 second session start experience for F2 capacity. I will update this thread once the fix is rolled out in few weeks.

Thank you

arjaano's avatar
arjaano
Advocate II
2 years ago

This should not be an accepted solution. The issue is still there. I'm on a F4 and facing 2+ minutes startup time.

  • wiegelman's avatar
    wiegelman
    Frequent Visitor
    2 years ago

    Agree, this should not be closed - we are also using F4 and the Spark Session is not starting, it is actually timing out for us...

    Diagnostics below:

    Diagnostic ID: 22a1d13c-d334-4828-8215-dfe8e65dfb60

    Timestamp: 2024-02-28T13:38:08.717Z

    Message: [object CloseEvent]

    JSON
    {
    "type": "close",
    "timeStamp": 731861.099999994,
    "code": 1000,
    "reason": "{\"reason\":\"Session error or stopped.\",\"state\":\"session-completed\"}",
    "wasClean": true,
    "target": {
    "protocolsProfile": [
    7,
    4173
    ]
    },
    "currentTarget": {
    "protocolsProfile": [
    7,
    4173
    ]
    },
    "isTrusted": true
    }

    Additional info: InstanceId: 2491b595-9a6b-4066-bd0d-0e41152e1754

    • alozovoy's avatar
      alozovoy
      Advocate II
      2 years ago

      This is a separate issue than the rest of in this thread. If you search the forums for "Livy session has failed" you will see a number of discussions.

    • wiegelman's avatar
      wiegelman
      Frequent Visitor
      2 years ago

      At least issue is acknowledged - hopefully gets fixed soon