PYSPARK notebook issue

Hi KUMARCH ,

The 430 error is a capacity exceeded message, so before anything else there are two specific things to check.

First, what SKU is assigned to your trial? Fabric trials can assign anywhere an F4 or F64 depending on the tenant. You can check this in Admin Portal > Capacity Settings. The SKU determines how many CUs you have available and how much a Spark session consumes at startup.

Second, and more importantly, check the Microsoft Fabric Capacity Metrics app. https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app

Even if the portal meter shows 0/4, you may have accumulated consumption over the past few hours and your capacity could be above 100% in throttling. Fabric uses a smoothing system where consumption is distributed across time windows, and if you have exceeded the limit the system blocks new operations until consumption drops back below the threshold, regardless of whether there are any active jobs at that moment.

If you open the Metrics app and see the CU bar at or above 100%, that is your problem. In that case you will have to wait until the percentage drops before trying again. My recommendation is not to attempt any new operations until you are below 50%, and from that point keep an eye on the Metrics app regularly to avoid hitting the limit again. Running Spark sessions without monitoring consumption is the fastest way to end up in the same situation.

If this response has been helpful, please don't forget to give it a Like and mark it as a Solution so other community members can find it easily.

Thank you!

Hi KUMARCH ,

 I ran into the same error today and thought of you.

In my case, I adjusted the Spark configuration. I had a Medium node running on an F4 capacity, and I changed it to Small, also adjusting the number of nodes. Give it a try and let us know if it works for you:

https://learn.microsoft.com/en-us/fabric/data-engineering/environment-manage-compute

If this response has been helpful, please don't forget to give it a Like and mark it as a Solution so other community members can find it easily.

Thank you!

This really sounds like a backend capacity or Spark allocation issue on the trial environment side, especially since even spark.range(5).show() is failing and the usage metrics are showing almost no activity. If regular Python notebooks still work, the problem is probably specific to Spark session provisioning rather than your notebook code.