Forum Discussion
Fabric Trial Capacity - Too Many Requests for Capacity Error
Since the start of this week, I am not able to run any notebooks inside Microsoft Fabric (F4 Trial Capacity). The default environment is configured with 8 spark driver and executor cores and 56 gigs of memory. So everytime I try to start a notebook session it returns a 430 error.
I can create an environment with 4 cores and 28gb memory and switch to the environment to start a session and run the notebooks but it takes 3-5 minutes for a session to start and my pipeline that runs notebooks one after the other keeps failing because the first notebook session does not close before the other one starts.
Has anyone else been facing similar issues in Fabric recently? Because it was working fine until last week (even though the trial capacity was F4)
6 Replies
- v-shchada-msftCommunity Support
Hi shashwotm,
Thank you for reaching out to the Microsoft Fabric Community Forum and thanks to the Tamanchu , arabalca for sharing helpful insights.
Just checking in, were you able to resolve the issue? If not please feel free to share an update and we will be happy to assist further.
Your feedback will also help others facing similar challenges.
Thank you. - TamanchuSuper User
Hi shashwotm,
This usually happens when the Fabric capacity has reached its Spark compute/concurrency limits, even on Trial capacities.
A few things worth checking :
- Go to Workspace Settings - Data Engineering/Science - Spark settings - Jobs to see active and queued Spark sessions.
- Sometimes orphaned Spark sessions remain active in the background and continue consuming capacity.
- The default Starter Pool can also be quite heavy for small capacities/trials.
A few practical mitigations that often help :
- Stop unused notebooks/sessions
- Reduce concurrent notebook executions
- Create a smaller custom Spark pool instead of using the default Starter Pool
- Enable High Concurrency mode when relevant
- Retry after the Spark session fully terminates
Microsoft documentation also explains this behavior here :
https://learn.microsoft.com/en-us/fabric/data-engineering/troubleshoot-permissions-capacityAnd for Spark concurrency limits :
https://learn.microsoft.com/en-us/fabric/data-engineering/spark-job-concurrency-and-queueingIf this helped, please consider marking the response as a solution so it can help others facing the same issue.
- arabalcaSuper User
Hi shashwotm ,
This error means your Fabric capacity has reached the concurrent Spark jobs limit. This is a SKU/capacity-level throttling issue, not necessarily a problem with your code.
Learn more here: https://learn.microsoft.com/fabric/data-engineering/spark-job-concurrency-and-queueing
I would recommend reviewing the following:
1. Review and cancel active jobs
Go to Workspace Settings → Data Engineering/Science → Spark settings → Jobs and cancel any non-critical Spark jobs.
There are often "zombie" notebook sessions consuming Spark slots without doing anything useful.
https://learn.microsoft.com/fabric/data-engineering/job-concurrency-queue-monitoring
2. Activate High Concurrency (if applicable)
This can help improve Spark session utilization and reduce concurrency-related issues.
3. Review Spark compute configuration
In fact, I ran into the same error today and immediately thought of you.
In my case, I adjusted the Spark configuration. I had a Medium node running on an F4 capacity, and I changed it to Small, also adjusting the number of nodes. That resolved the issue for me.
Give it a try and let us know if it works for you:
https://learn.microsoft.com/en-us/fabric/data-engineering/environment-manage-compute
Additional references that may help:
https://learn.microsoft.com/fabric/data-engineering/job-queueing-for-fabric-spark
https://learn.microsoft.com/fabric/data-engineering/autoscale-billing-for-spark-overview
If this response has been helpful, please don't forget to give it a Like and mark it as a Solution so other community members can find it easily.
Thank you!
- tayloramySuper User
Hi shashwotm,
This is a known limitation, but a fix is coming very soon!
Check it out: Custom Live Pools for Fabric Data Engineering (Pre... - Microsoft Fabric Community - GilbertQSuper User
Hi shashwotm
If your requirements are not too large in terms of data processing or you can let it run for longer. why not change the size of a spark pools to even a smaller size which will then allow it to run successfully on your smaller fabric capacity. So make sure to ensure that what you're running in Spark is using the optimal code, and that to ensure once again that it is not using as much of the Spark resources as needed.
- v-shchada-msftCommunity Support
Hi shashwotm
Just checking in to see if your issue has been resolved. Please let us know if you need any further assistance.
Thank you.