Forum Discussion
Solved
Livy session becomes dead when reading large file in Fabric Notebook
Hi everyone, I'm running a notebook in Microsoft Fabric where I read a large file, around 2GB. The notebook takes a long time to execute, and eventually I receive the following error: InvalidHttpR...
Hello gbetancurm
It is probably a driver out-of-memory issue, which would happen if you're using Pandas dataframe instead of Spark. Pandas is not distributed by design, so none of the executors are used and the driver gets overloaded.
You can start by looking at the Spark application details > Logs > Driver - you should find additional details on why the application failed.
Same error here, but in synapse analytics.