Spark Jobs - Monitor Page

This is mostly a quirk of how the Spark History Server reports state, not that the queries are truly still running.

When a streaming job is cancelled, the driver and executors are stopped, but the Structured Streaming UI (in the history server) is based on last known state written to event logs. If the query did not shut down cleanly (for example, abrupt cancel), its status can remain as “RUNNING” and the duration counter continues to tick because there is no final termination event recorded. So what you are seeing is a stale UI state, not an active computation.

If they were actually running, you would see active Spark applications consuming cluster resources in the workspace or capacity metrics. To be sure, check the Fabric/cluster monitoring or active Spark sessions. There is nothing to stop at this point because the compute is already terminated. If you want to avoid this in future, ensure graceful shutdown of streaming queries (for example, stop() on the query or checkpoint-aware termination) so Spark can persist a proper “TERMINATED” state.