Forum Discussion
Memorable MemberAnother meaningless spark error: Session is unable to register ReplId
There is a new error today. This beauty is in the stderr of the driver:
Session is unable to register ReplId: default
It is frustrating to use this pyspark environment. Every pyspark environment from Microsoft seems worse than the previous. HDInsight is wonderful, Synapse was pretty bad, and Fabric is terrible. ... I'm opening spark support tickets with Mindtree at a faster rate than they are going to Microsoft. Since I already have several tickets without an ICM, I'm not eager to continue creating these. But I want to put this here so it can be googled, at the very least.
The notebook session won't be created. The most prominent error stack in stderr of the driver looks like so:
2025-01-15 10:42:14,038 INFO ApplicationMaster [shutdown-hook-0]: Unregistering ApplicationMaster with FAILED (diag message: User class threw exception: java.lang.Exception: Session is unable to register ReplId: default
at org.apache.livy.repl.ReplDriver.getOrCreateSession(ReplDriver.scala:419)
... it is presented as a WARN in the logs, but the ultimate impact is that my notebook won't start.
Earlier in the stderr, there is a log output with the ERROR severity, but it is even less meaningful than the WARN:
2025-01-15 10:42:12,846 ERROR ServerRuntime$Responder [SparkUI-85]: An I/O error has occurred while writing a response message entity to the container output stream.
Earlier than that are some other WARN's, not sure if they are relevant:
If anyone has seen these errors, or if you are already working on a support ticket please let me know. I will be very grateful if you can spare me a week or two of effort.
13 Replies
Hi dbeavon3,
Thank you for reaching out to Microsoft Fabric Community Forum.
Please verify if there are any known issues or bugs with the specific version of Microsoft Fabric.
If there are no bugs, please try again later; it might be a temporary glitch with the backend services.
Did I answer your question? Mark my post as a solution, this will help others!
If my response(s) assisted you in any way, don't forget to give "Kudos"Regards,
Vinay Pabbu
dbeavon3Anonymous
Thank you for the reply.
In this case, I seem to be the first one to ever post about this message to the public internet.Regardless if it is transient or not, customers need to understand the meaning of any error message. We need to know if errors can be fixed in our own custom code or must be fixed in Microsoft's code. We need to know if the fix will be considered a "workaround" or a "permanent" fix. We need to know if retries should be done x3 times or x30 times. We need to know if the problem is affecting all customer tenants, or if they are isolated to just our own customer tenant.
It is extremely unlikely that this is a "temporary glitch" which I will never see again. I should mention that I've only been using Fabric Spark for a few days, and I've encountered many, many errors which are meaningless to me. I have experience with other Spark platforms and in most cases we are able to google for an explanation about any error message. In most Spark platforms the error messages can be fixed by correcting our own code. However the bugs I'm seeing in this Fabric flavor of Spark seem to originate from Microsoft's home-grown extensions to Spark. (like Livy, for example)
If you are able to interpret this error message or the callstack please let me know. You could save me a LOT of time. The Mindtree support cases take many weeks, while waiting on a response from the back-end PG. I have started that process, but nobody from Microsoft has engaged with me yet.
Hi dbeavon3,
I apologize for the challenges you're facing. I understand the importance of having clear explanations and resolutions for error messages. I suggest raising a support ticket. If you have already done so, I kindly request you to wait for the response.
How to create a Fabric and Power BI Support ticket - Power BI | Microsoft LearnRegards,
Vinay Pabbu
