Forum Discussion
Notebook Error: SparkCoreError/PersonalizingFailed: Livy session has failed
Hi mdhewitt83 . I got the same issue today. I have pipeline, which use environment, where we have custom library ( inside we use great expectations ). Before pipeline worked fine and today I got error :
- in pipeline : Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Failed to create Livy session for executing notebook.
- when running notebook itself : Error code: SparkCoreError/PersonalizingFailed.
I got the same issue starting the last evening, nothing has changed today. I've just looked into the environment libraries, changed the version of libraries to the newer one and published it (in my case it was ms-graph). After it, it started to work. It's really annoying when such an issue is happening in your production environment.
I just tried to update library ( azure_cli from version 2.61.0 to 2.63.0 ) by adding new .whl file to my environement. Now I got an issue during publishing : Unknown error occurred while publishing.
In my case your resulution doesn't work SirJudzin
Hi m98ateusz
Does your environment run on Runtime 1.3? Recently there is a known issue related to Runtime 1.3 when custom libraries are installed. Known issue - Fabric Runtime 1.3 causes invalid libraries - Microsoft Fabric | Microsoft Learn
Best Regards,
JingHi,
I did once again thing which SirJudzin did.
First time I planned to do it in two steps :
1.) Add new version of library (whl file)
2.) Delete old version of library (whl file).
However it failed during first step, so I decide to do it in one step :
1.) Add new version of library and delete old old version
and it worked.
My runtime version is 1.2