Forum Discussion

lsabetta's avatar
lsabetta
Frequent Visitor
9 months ago
Solved

Automatically end session configuration

Hi community,   I have a pipeline with 2 Notebooks with 1 CopyData between both. I turned on the spark high concurrency session for pipelines and choose a name for a session tag in the Notebooks. ...
  • v-tejrama's avatar
    v-tejrama
    9 months ago

    Hi lsabetta ,

     

    Thanks for sharing this. I understand how frustrating it can be when the Automatically end session setting doesn’t stay at the value you configure. What you’re seeing is actually a known limitation in Microsoft Fabric right now. When you update that setting to anything above the default one minute, the change doesn’t persist after a page refresh or workspace reload, which makes it look like it’s reverting back.

    This isn’t a configuration mistake on your end it’s related to how the current Fabric interface handles session timeout values for Spark. The team is aware of this behavior, and it’s expected to be addressed in a future update. For now, you can safely rely on enabling High Concurrency mode at the workspace level and use the same session tag across your notebooks. That setup ensures that the same Spark session is reused during your pipeline execution, even if the timeout display resets in the interface.

    If your Copy Data activity is taking several minutes and you notice a new session being created despite high concurrency being enabled, it would be worth confirming that both the workspace-level concurrency setting and the session tag are consistently applied. If everything looks correct but the behavior continues, I’d suggest opening a support ticket with Microsoft Fabric Support so they can verify whether your workspace is affected by this persistence issue.

    You can also refer to the official documentation on configuring high concurrency mode for notebooks and pipelines here:
    https://learn.microsoft.com/en-us/fabric/data-engineering/configure-high-concurrency-session-notebo…


    Best Regards,
    Tejaswi.
    Community Support