We have had several occurences where a Notebook will exit the notebook code, but the Spark session is inexplicably not closed.
This causes us to incur signifigant capacity consumption until an admin recognizes that a notebook has been running for much longer than expected via the Monitor tool or from Capacity alerts and manually ends the session. However, by the time corrective action has been taken, we have already utilized more capacity than we should have, leading to further potential issues like throttling.
Please add the ability to add an optional timeout for Notebooks that are run in the background. Once the time limit has been reached, the notebook should terminate and the spark session should be ended.
on:
RDL Subscription Options in Power BI Service