We have had several occurences where a Notebook will exit the notebook code, but the Spark session is inexplicably not closed.
This causes us to incur signifigant capacity consumption until an admin recognizes that a notebook has been running for much longer than expected via the Monitor tool or from Capacity alerts and manually ends the session. However, by the time corrective action has been taken, we have already utilized more capacity than we should have, leading to further potential issues like throttling.
Please add the ability to add an optional timeout for Notebooks that are run in the background. Once the time limit has been reached, the notebook should terminate and the spark session should be ended.
No CommentsBe the first to comment
Recent ideas
Allow External OneLake Data Shares to Be Accepted into Multiple Fabric Item Types
Fabric External Data Sharing provides an excellent zero-copy mechanism for sharing OneLake data across tenants. However, the receiving tenant currently has to accept the share into a Lakehouse. I w...Mourak23 hours agoNew MemberNew5Views0likes0CommentsAllow OneLake Shortcut Authentication Mode to Be Changed In Place
OneLake shortcuts support passthrough and delegated authentication, but changing the authentication model currently requires deleting and recreating the shortcut. I would like shortcut authenticatio...Mourak23 hours agoNew MemberNew7Views0likes0CommentsTenant-Wide OneLake Shortcut Lineage and Impact Analysis
This is grounded in a documented limitation: shortcut lineage is currently workspace-scoped, and Microsoft specifically says lineage for shortcuts to warehouses and semantic models isn't currently av...Mourak23 hours agoNew MemberNew16Views0likes1CommentAdd an End-to-End Production Reference Architecture for Microsoft Fabric Real-Time Intelligence
I would like to see a complete end-to-end Microsoft Fabric Real-Time Intelligence reference architecture that demonstrates how the individual RTI capabilities work together in a production-oriented s...Mourak23 hours agoNew MemberNew5Views0likes0CommentsImprove CI/CD consistency for Microsoft Fabric Real-Time Intelligence Eventstreams
Microsoft Fabric Real-Time Intelligence has made strong progress with Git integration and deployment pipelines, but lifecycle management can still be inconsistent across Eventstream sources, destinat...Mourak23 hours agoNew MemberNew3Views0likes0Comments