Error while creating the livy session & Failed to create session for executing notebook

Hi RaivisMezis ,

Thank you for reaching out to the Microsoft fabric community forum.

To troubleshoot the Livy session failures in your Microsoft Fabric pipeline, start by reviewing how session tags are used. Since your pipeline runs ~10 notebooks concurrently each triggering ~30 others via RunMultiple DAG and all use the same session tag, this can overload the Live Pool Manager (LPM), especially in the North Europe region where throttling issues have been confirmed. Assigning unique session tags or reducing concurrency can help.

Also, avoid setting spark.fabric.pools.skipStarterPools unless necessary, as skipping starter pools increases session creation time and failure risk. Use built-in notebook utilities like notebookutils.stop() and restartPython() to manage sessions cleanly.

For diagnostics, run the LPM throttling query to identify overloaded nodes and coordinate with the LPM team if needed. Since manual notebook runs work fine, the issue likely lies in how Fabric orchestrates sessions under pipeline load.

Hope this helps, feel free to reach out for any further questions.

Thank you.