topic Re: Livy session becomes dead when reading large file in Fabric Notebook in Data Engineering

Livy session becomes dead when reading large file in Fabric Notebook

gbetancurm — Mon, 23 Feb 2026 14:07:14 GMT

Hi everyone,

I'm running a notebook in Microsoft Fabric where I read a large file, around 2GB.

The notebook takes a long time to execute, and eventually I receive the following error:

InvalidHttpRequestToLivy: Session -------- of workspace --------- on pool -------- is not running. It is either completed or not fully yet ready. Scheduler state : Scheduled. Plugin state : Cleanup. Livy state : dead HTTP status code: 400. Trace ID: --------.

My question is: Can this issue be handled from the code side, or is it more likely related to Spark/Livy configuration settings? What would be the best way to fix it?

Has anyone experienced something similar when processing large files in Fabric?

Any guidance would be greatly appreciated.

Thanks in advance!

Re: Livy session becomes dead when reading large file in Fabric Notebook

deborshi_nag — Tue, 24 Feb 2026 09:55:10 GMT

Hello @gbetancurm

It is probably a driver out-of-memory issue, which would happen if you're using Pandas dataframe instead of Spark. Pandas is not distributed by design, so none of the executors are used and the driver gets overloaded.

You can start by looking at the Spark application details > Logs > Driver - you should find additional details on why the application failed.

Re: Livy session becomes dead when reading large file in Fabric Notebook

joaotinti — Tue, 24 Feb 2026 15:56:07 GMT

Same error here, but in synapse analytics.

Re: Livy session becomes dead when reading large file in Fabric Notebook

v-veshwara-msft — Fri, 27 Feb 2026 09:51:01 GMT

Hi @gbetancurm ,

Thanks for reaching out to Microsoft Fabric Community.

Just checking in to see if the response provided by @deborshi_nag was helpful.

If you have reviewed the Spark application Driver logs as suggested and still need further assistance, please feel free to share additional details so we can assist further.

Thank you.

Re: Livy session becomes dead when reading large file in Fabric Notebook

v-veshwara-msft — Tue, 03 Mar 2026 04:20:24 GMT

Hi @gbetancurm ,

We wanted to kindly follow up regarding your query. If you need any further assistance, please reach out.
Thank you.