Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

Livy session has failed. Session state: Dead,

Hi Fabricators!   Our daily run gets sometimes hit by the following error while trying to run mssparkutils.notebook.runMultiple():   Notebook execution failed at Notebook service with http status...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Anonymous ,
    This error usually means the Spark session was not fully active when mssparkutils.notebook.runMultiple() was triggered.This might due to idle timeouts, especially when the pipeline runs after a period of inactivity.To reduce failures, you may try adding a small command like spark.range(1) at the start of the parent notebook to initialize the session, and if you are launching several notebooks at once, consider staggering them slightly. Also, reviewing your Spark pool’s min/max settings may help reduce session startup delays.
    Please refer the links below for detailed infromation:
    https://learn.microsoft.com/en-us/fabric/data-engineering/microsoft-spark-utilities
    https://learn.microsoft.com/en-us/fabric/data-engineering/get-started-api-livy-session 


    I hope this resolve your query.If so,give us kudos and consider accepting it as solution.

    Regards,
    Pallavi.