Forum Discussion

akanane's avatar
akanane
Frequent Visitor
2 years ago
Solved

Can't stop spark session using mssparkutils

  Hello,   I get the following error while trying to stop a spark session in a notebook using: mssparkutils.session.stop()   ----------------------------------------------------------...
data engineering
  • v-cboorla-msft's avatar
    v-cboorla-msft
    2 years ago

    Hi akanane 

     

    Apologies for the issue you have been facing. Currently mssparkutils.session.stop() does not support in Fabric for now. The feature is releasing. We will have to wait until mssparkutils environment utils is ready.

     

    Appreciate your patience.

     

    Hope this helps. Please let us know if you have any further questions. Glad to help.

v-cboorla-msft's avatar
v-cboorla-msft
Microsoft Employee
2 years ago

Hi akanane 

 

Apologies for the delay in response.

Currently mssparkutils supports only the below utilities.

fs:Utility for filesystem operations in Fabric
notebook:Utility for notebook operations (e.g, chaining Fabric notebooks together)
credentials:Utility for obtaining credentials (tokens and keys) for Fabric resources
lakehouse:Utility for lakehouse operations (e.g, create, delete, update, list lakehouse)

It doesn't support the session utility, this is the reason why you are facing an issue while running the code "mssparkutils.session.stop()".

You can use spark.stop() :

If you have spark as a SparkSession object then call spark.stop() to stop the session. This ensures that resources are properly released and the Spark application terminates gracefully.

 

I hope this helps. Please do let us know if you have any further questions.

 

v-cboorla-msft's avatar
v-cboorla-msft
Microsoft Employee
2 years ago

Hi akanane 

 

Apologies for the issue you have been facing. Currently mssparkutils.session.stop() does not support in Fabric for now. The feature is releasing. We will have to wait until mssparkutils environment utils is ready.

 

Appreciate your patience.

 

Hope this helps. Please let us know if you have any further questions. Glad to help.

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi akanane 

     

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.

    In case if you have any resolution please do share that same with the community as it can be helpful to others.
    Otherwise, will respond back with the more details and we will try to help .

     

    Thanks

    • akanane's avatar
      akanane
      Frequent Visitor
      2 years ago

      Hello,

       

      Thank you for your answer, spark.stop() works 🙂 (but we don't see the session stopped in the UI)

      • v-cboorla-msft's avatar
        v-cboorla-msft
        Microsoft Employee
        2 years ago

        Hi akanane 

         

        Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.