Forum Discussion
Advocate IFabric Notebook Stuck in "Running" Status
Hi smoqt,
Thank you for your follow-up and the update provided.
-
At present, I have not found any documentation to confirm that this is a bug.The underlying cause seems to be related to improper token caching logic during Spark session initialization, as well as missing runtime dependencies (such as ZooKeeper classes) in the driver environment.
-
Pipeline activity timeouts do not terminate the associated Spark job. Consequently, the notebook may continue to run in the background, consuming Capacity Units (CUs). To manage this, we recommend using the "Monitor Activities" feature in Microsoft Fabric, as detailed in the Monitor activities in Microsoft Fabric - Training | Microsoft Learn, to track and manually terminate long-running jobs. Additionally, you may consider implementing a watchdog script leveraging the REST API to monitor and cancel jobs exceeding a predefined runtime.
-
Please utilise the Capacity Metrics app to identify notebooks that have high CU consumption. It is advisable to break lengthy notebooks into smaller, modular units to minimise the risk of idle drivers. Furthermore, configuring the Spark job admission rules to limit concurrency may help prevent system overload.
Given the systemic nature of this issue and the potential cost implications, we recommend raising a support ticket with Microsoft for a thorough investigation and resolution. You may raise a ticket via the Microsoft Fabric Support and Status | Microsoft Fabric.
Additionally, please refer to the following links:
What is the Microsoft Fabric Capacity Metrics app? - Microsoft Fabric | Microsoft LearnUnderstand the metrics app compute page - Microsoft Fabric | Microsoft Learn
Install the Microsoft Fabric capacity metrics app - Microsoft Fabric | Microsoft LearnIf you find our response helpful, kindly mark it as the accepted solution. This will assist other community members facing similar queries.
Should you have any further questions, please feel free to reach out to the Microsoft Fabric community.
Thank you.
-
Advocate II appreciate the response, but I still have some concerns and would like clarification on a few points:
Is this a confirmed and documented bug?
If so, could you please share a link or internal reference? I found a nearly identical issue discussed on Reddit, which makes me think this may be systemic.Error-handling logic doesn't apply here.
In my case, no error was thrown in the notebook itself. The only errors were in the driver node logs (e.g., token caching failure and NoClassDefFoundError), which were not surfaced to the notebook runtime. So retry logic wouldn’t help, since the notebook never technically failed.The notebook already runs during low-usage hours.
It is scheduled to run early Sunday mornings, which should already avoid peak concurrency.Regarding pipeline timeouts:
If I wrap the notebook in a pipeline with an activity timeout, will that actually terminate the Spark job? Or will the notebook continue to run indefinitely in the background, consuming Capacity Units (CUs)? This is critical for cost management.
My biggest concern is the potential for runaway CU consumption with no clear failure or timeout mechanism. Any guidance on how to monitor or guard against this scenario—especially when Spark stalls before launching any tasks—would be appreciated.
