Forum Discussion
Shared Sessions are taking longer to get attached while executing notebook in pipeline run in fabric
Hello,
Over the past 3–4 days, we've observed an increase in the time it takes for a shared Spark session to attach when transitioning from one notebook activity to another. We've provided screenshots comparing previous runs with the latest ones.
While the duration of the middle activity remains mostly unchanged, the start and end activities are taking noticeably longer than before. Although the time difference may appear minor, it's becoming more significant—especially when this logic is executed multiple times. These extra minutes are adding up and contributing to a longer overall runtime.
before:
after:
If anyone is aware of this issue or has any insights or suggestions, your help would be greatly appreciated.
Thanks in advance, and have a great day!
The screenshots show increased attach times for shared spark sessions in fabric pipelines esp., at the start and end notebook activities. This slowdown is likely due to backend resource contention/session reuse delays within the shared spark pool. When multiple notebooks run concurrently or capacity is under load, session warmup and context switch times may increase.
actions that you can try:
Monitor fabric capacity metrics (CPU, memory, spark concurrency).
Try assigning a dedicated spark session per pipeline or using “Run in new session” for critical notebooks.
Clear idle sessions periodically using the fabric REST API or capacity admin tools.
Raise a fabric support ticket with timestamps for correlation as this might be a transient backend performance regression.
4 Replies
The screenshots show increased attach times for shared spark sessions in fabric pipelines esp., at the start and end notebook activities. This slowdown is likely due to backend resource contention/session reuse delays within the shared spark pool. When multiple notebooks run concurrently or capacity is under load, session warmup and context switch times may increase.
actions that you can try:
Monitor fabric capacity metrics (CPU, memory, spark concurrency).
Try assigning a dedicated spark session per pipeline or using “Run in new session” for critical notebooks.
Clear idle sessions periodically using the fabric REST API or capacity admin tools.
Raise a fabric support ticket with timestamps for correlation as this might be a transient backend performance regression.
Hi YashRaj5,
Thank you for reaching out to Microsoft Fabric Community.
Thank you Vinodh247 for the prompt response.
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.
Thanks and regards,
Anjan Kumar Chippa
