Forum Discussion
Advocate IIHow many sessions I could run in F2 Capacity?
Hi AnmolGan81 ,
Thanks for the detailed update and for sharing what you’ve observed, it’s great that you’ve figured out what’s happening with the sessions.
The issue seems to be:
On an F2 SKU, the limited resources (4 Spark vCores) mean you can only run one Spark session at a time, regardless of how many notebooks you’re using. So, if there’s an active session in one notebook, trying to start another session in a different notebook results in the "too many requests" error.
You’re also right that just closing the notebook or stopping the job from the Monitor doesn’t fully terminate the session unless it times out or you explicitly stop it.
A Few Suggestions:
-
Ensure the Session is Stopped:
-Adding spark.stop() to the end of your notebook is the reliable way to free up resources.
-Restarting the capacity works, but that’s more of a workaround than a long-term fix. -
Enable Bursting (with Limitations):
-Bursting temporarily increases the available Spark vCores for your capacity (e.g., an F2 SKU can scale up to 20 Spark vCores during bursts).
-This allows for better concurrency, meaning you may be able to run multiple notebooks simultaneously during a burst period, provided the combined workload does not exceed the burst limit.
-However, bursting is not a permanent solution and can only support short-term spikes in usage. If both jobs are resource-intensive, you might still run into resource contention even with bursting enabled.
-Bursting depends on resource availability, and it's important to note that it only helps with concurrency but doesn't guarantee success for all parallel workloads. -
Upgrade Your Capacity:
-If running multiple sessions in parallel is essential, upgrading to an F4 SKU or higher would provide more Spark vCores and better concurrency support.
Let me know if you need help managing sessions, enabling bursting, or exploring capacity options. Happy to assist further!
If this helps, please accept as solution to help others benefit, a kudos would be appreciated.
Best regards,
Vinay.-
Advocate IIWhen I try to get into the notebook and try to run the same query I get below error, and nothing is running in job monitor as I checked before running any queries.
JSON { "type": "close", "timeStamp": 108670.20000001788, "code": 1000, "reason": "{\"reason\":\"Session error or stopped.\",\"state\":\"session-completed\"}", "wasClean": true, "target": { "url": "wss://6dc6322c24a042a2828f8e2aa68f9b82.pbidedicated.windows.net/webapi/capacities/6DC6322C-24A0-42A2-828F-8E2AA68F9B82/workloads/Notebook/Data/Direct/api/workspaces/cf8788ae-d2a9-4176-8900-5299acf0cce7/artifacts/0465d3f7-5fa7-4b86-a091-1395eb545a70/jupyterApi/versions/1/api/kernels/abd3f4ed-1da3-4124-a965-ae10fc877312/channels?token=dummy_token&session_id=c9350f12-d2b1-470a-ab6c-a33f1394c6aa", "readyState": 3, "protocolsProfile": [ 7, 3975 ] }, "currentTarget": { "url": "wss://6dc6322c24a042a2828f8e2aa68f9b82.pbidedicated.windows.net/webapi/capacities/6DC6322C-24A0-42A2-828F-8E2AA68F9B82/workloads/Notebook/Data/Direct/api/workspaces/cf8788ae-d2a9-4176-8900-5299acf0cce7/artifacts/0465d3f7-5fa7-4b86-a091-1395eb545a70/jupyterApi/versions/1/api/kernels/abd3f4ed-1da3-4124-a965-ae10fc877312/channels?token=dummy_token&session_id=c9350f12-d2b1-470a-ab6c-a33f1394c6aa", "readyState": 3, "protocolsProfile": [ 7, 3975 ] }, "isTrusted": true } Additional info: InstanceId: e529904c-127d-4fe7-bcfd-99e39c992504
Community SupportHi AnmolGan81 ,
Thanks for sharing the error details. Based on the information, it seems that the session might have timed out, been forcibly stopped, or encountered an issue while maintaining resources.
Recommended Actions:
-
Restart the Spark Session:
You can try restarting the session in your notebook by going to Connect > New standard or High Concurrency session. This will create a new session and re-establish the connection.
Alternatively, you can stop all operations, close the notebook, and open it again to start with a fresh session
If the issue persists, you can try enabling Bursting and Auto-scaling as suggested in the previous response.
Hope these help. Please reach out if you face any issues.
If this helps, please accept as solution to help others benefit, a kudos would be appreciated.
Best regards,
Vinay.