Forum Discussion

Mirdula's avatar
Mirdula
Frequent Visitor
1 year ago
Solved

Notebook Status - Stopped (Session Timed Out)

Hello everyone,  Would appreciate your thoughts on the below.    I had a notebook in the bronze layer, which I used to ingest data from an ERP system. Later, I created a duplicate of this file and...
workspaces
  • Rufyda's avatar
    Rufyda
    1 year ago

    Hi Mirdula 😊

    Thank you for your detailed feedback.

    What you're experiencing is quite common in Spark-based environments like Microsoft Fabric. Here's a quick clarification that may help:

    When a notebook completes all cells successfully, but the Spark session doesn't close explicitly, the Fabric platform sometimes registers the session as "Stopped" instead of "Success" — even though no error occurred.

    What you can try:

    To ensure the session ends gracefully, you can manually stop the Spark session at the end of your notebook using:

     

    spark.stop()

     

    This helps Fabric detect that the notebook has completed intentionally, not due to timeout or inactivity.

    Also, adding a cell like this at the end of your notebook might help signal a successful end:

     

    print("Notebook completed successfully.")

     

    While this doesn't guarantee a “Success” label in every case (as it depends on backend handling), it improves the chances — and keeps your logs cleaner.

    Let me know if that helps!
    And if the previous suggestion worked partially, feel free to mark it as solution and give a Kudo if it helped. 😄

Rufyda's avatar
Rufyda
Super User
1 year ago

 

1. Check Spark Session Timeout Settings:
Even though the Spark session timeout is set to 30 minutes, sometimes the default Spark configuration can cause the session to timeout if there is no activity (like reading or writing data). You can verify and adjust your session settings to ensure they align with the specific workload you're running.

How to Adjust Spark Timeout Settings:

Try modifying the Spark configuration within your notebook to allow for more leniency during the runtime.

Use the following command to explicitly set the timeout settings within the notebook itself:

 

spark.conf.set("spark.sql.broadcastTimeout", "3600") # Set timeout to 1 hour spark.conf.set("spark.network.timeout", "3600") # Set network timeout to 1 hour

If this solution works for you, feel free to like it and confirm that it’s suitable for your case.

Mirdula's avatar
Mirdula
Frequent Visitor
1 year ago

Hi Rufyda, 

 

Thank you for your response. and now I can see that the status in the recent run list shows "stopped". My question is, earlier it showed "success" and from yesterday it showed "stopped (session timed out)" and after applying the code you gave me it shows "stopped". 

Is there a way to fix this to show "success" just like before? Have you faced anything similiar? 🙂