Forum Discussion
Memorable MemberAnother meaningless spark error: Session is unable to register ReplId
Memorable MemberFYI, I closed the ticket, and got a very low level of support for the issue.
It turned out that VM's weren't being started or something like that. I'm guessing that it is NOT a problem that can be addressed by the Fabric-Spark team themselves. It is probably a cross-team problem. These are the types of issues which Mindtree support is probably not equipped to work on. I think they can hardly work on a bug that is isolated to the Fabric-Spark team, let alone one that involves VM provisioning in Azure.
My biggest complaint is that the error message surfaced to the customer is meaningless. There are back-end logs (kusto?) where the engineers can dig into the failures in the cluster itself; but of course they don't allow customers to perform those investigations when our workloads are dying. So we can't independently investigate these bugs, and if we ask Mindtree to do it, they can't either. You have to wait about two weeks for the PG (Microsoft itself) to get on board.
I suspect this bug is one that customers will see more frequently on certain azure regions. Eg. we are hosting on North Central US. I don't think there will ever be a permanent fix, but hopefully they will tweak the error messages that are surfaced to their customers one day:
Below is the minimal explanation of what can be seen in their kusto logs, when this bug is happening in our Spark jobs on Fabric.
Hi dbeavon3,
Thank you for providing detailed information from the support team. It will be helpful for other members of the community who have similar problems to solve them faster.
I suggest submitting your detailed feedback and ideas through Microsoft's official feedback channels, such as Microsoft Fabric Ideas. Feedback submitted through these channels is frequently reviewed by the product teams and can contribute to meaningful improvements.
https://ideas.fabric.microsoft.com/ideas/search-ideas/
Regards,
Vinay Pabbu
- dbeavon3
I wish the normal support channels were more effective. I'm certainly not the only customer who encountered this bug in Spark, yet it feels like I'm fending for myself. The Microsoft PG won't often fix bugs right away, nor publish the details to their "known issues" list.
This community is at least one place we can raise awareness about bugs.
The only problem with this community is that I do NOT necessarily want to be the one providing future support to the other customers, after they encounter the same issue and fail to get meaningful support from Microsoft.
Hi dbeavon3,
Thanks for sharing the update from supoort ticket. At this time we are closing this thread. If you have any further issues, please start a new thread in the community forum, and we are here to assist you. Thankyou for your understanding and continuous support.
Thank you for being part of the Microsoft Fabric Community.Regards,
Vinay Pabbu