Forum Discussion
Queued time is elevated - can this trigger a notebook failure?
Hi dbeavon3 ,
Thank you for the detailed information and understanding you provided. I apologize for any inconvenience caused.
As you mentioned, you are facing an issue with "Session is unable to register ReplId". This appears to be a new issue for Microsoft, and we appreciate you bringing it to our attention.
To ensure this is addressed effectively, I recommend you submit a support ticket directly to Microsoft using this link: https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket
Thankyou.
Hi dbeavon3
Thank you for reaching out to the Microsoft fabric community forum.
Yes, elevated queued time can potentially trigger a notebook failure.
It appears that the issue is related to a delay in resource allocation due to autoscaling, which triggered Livy’s timeout mechanisms and caused a failure. To address this, I recommend adjusting the autoscaling configuration and reviewing Livy’s timeout settings.
I am including a simpler thread that has already been resolved. Please review it to gain a better understanding.
https://community.fabric.microsoft.com/t5/Data-Engineering/notebook-queued-time/m-p/4103235
I hope my suggestions give you good ideas, if you need any further assistance, feel free to reach out.
If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
Hi Anonymous
Thank you for your time. You seem to be familiar with the Fabric flavor of Spark (and that is not a very common thing). I have an ongoing "pro" support case underway for almost a week, and nobody knows what the platform is doing. (The engineers have opened a "collab" with the ADF/pipeline team; and I'm guessing that the ADF team will cause another three-day delay for no good reason....)
It is disappointing to hear your conclusion that Livy timeout mechanisms might be causing failures. Do you have any reference material that describes the ten minute timeout? The docs you shared indicate that livy jobs can remain queued up to 24 hours. It is not ten mins, from what I can see.
The folllowing link describes a queue restriction that is based on the number of jobs that can be queued (the max time is always 24 hours)
Concurrency limits and queueing in Apache Spark for Fabric - Microsoft Fabric | Microsoft Learn
... in any case I don't think this is relevant to my error since any type of a queue capacity restriction would give a pretty obvious error message to the user. Whereas the error I'm seeing is an unfriendly error message in the stderr of the driver (ie. "Session is unable to register ReplId: default"). It has the feeling of a bug.
Ten minutes is extremely short. Especially if there is no visibility for customers to see the backlog growing on the cluster. If we could review these lifecycle operations in the cluster with our own eyes, then it would be easy to correlate the notebook failures to whatever is happening in the cluster. But in Fabric they won't give any surface area to monitor the cluster, because they want to make things "easy".
In some of the other Livy implementations (Synapse and HDI) the ten minutes on the livy queue would not cause any failure. In fact we could send dozens of jobs that take 30 mins each to complete (five running at a time), and we never have to worry about these livy timeouts!
Let me know if you can share docs that describe the error (Session is unable to register ReplId)
I would not be spending an entire week working with with a half-dozen engineers at Mindtree, if the error simply explained that there was a livy timeout of some kind. Eg. "A Livy timeout (10 mins) is preventing any additional notebooks from being executed at this time. Please try submitting your notebook to this pool on a different day."
Hi dbeavon3 ,
Thank you for the detailed information and understanding you provided. I apologize for any inconvenience caused.
As you mentioned, you are facing an issue with "Session is unable to register ReplId". This appears to be a new issue for Microsoft, and we appreciate you bringing it to our attention.
To ensure this is addressed effectively, I recommend you submit a support ticket directly to Microsoft using this link: https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket
Thankyou.
Hi Anonymous
Yes there is a support ticket. Those take some time and effort. On rare occasion I can get more timely support here, than with Mindtree & Microsoft "pro" support.
My only goal for the support ticket is to get this issue added to the "known issues" list. If you never see it on that list, then you will know that I have failed.Hi dbeavon3 ,
We apologize for any inconvenience this may cause and appreciate your patience as we work to resolve the issue. Microsoft engineers are actively investigating to find a solution.
We understand that support tickets require time and effort, and we value your cooperation.
Our priority is to ensure this issue receives the necessary attention. While we cannot guarantee immediate inclusion on the "known issues" list, please be assured that we are working closely with the support teams to escalate its visibility.
If you have any further concerns, please feel free to reach out.
Thank you for being a valued member of the Microsoft Fabric Community!