topic Re: Not able to run single command over the notebook in Data Engineering

Not able to run single command over the notebook

AnmolGan81 — Tue, 07 Jan 2025 07:27:04 GMT

Hi I have been trying to simple load data from an already present table in my lakehouse, but whenever I create notebook and right click on the present delta table and try to run the command I get below error:

We are unable to start a notebook session. To re-establish the connection, run the notebook again.Diagnostic info: (fetch_kernel_specs; p9wwvu) [0] null

Below command I am trying to run

df = spark.sql("SELECT * FROM Lakehouse1.publicholidays LIMIT 1000")

display(df)

I have fabric capacity of F2 sku but still not able to run anything over notebook, does anyone else faced this issue?

Re: Not able to run single command over the notebook

jennratten — Tue, 07 Jan 2025 12:52:35 GMT

What happens when you manually try to to start a new session? On occassion I will get a similar error - when I reconnect the session and run the notebook it resolves.

Re: Not able to run single command over the notebook

AnmolGan81 — Tue, 07 Jan 2025 13:13:22 GMT

That gives the same error as posted most of the time I got below error for F2 capacity, when I am only executing query to create a delta table from the file, I dont know how many concurrent session an F2 Capacity can run, also I did not find any documentation related to it:

Message

This spark job can't be run because you have hit a spark compute or API rate limit. To run this spark job, cancel an active Spark job through the Monitoring hub, choose a larger capacity SKU, or try again later. HTTP status code: 430 {Learn more}

Error Code

TooManyRequestsForCapacity

Re: Not able to run single command over the notebook

Anonymous — Thu, 09 Jan 2025 12:18:22 GMT

Hi @AnmolGan81,

Thank you for reaching out to the Microsoft Fabric community forum.

The issue appears to be that the F2 SKU has limited resources, with only 4 Spark vCores available. This means you can only run one Spark session at a time, even if you're using multiple notebooks. If there's an active session in one notebook, attempting to start a new session in another notebook causes the error.

To fix this, you have to restart the capacity and manually terminate the Spark session from the notebook. Once that's done, you can easily restart the session whenever needed.

I hope my suggestions give you good ideas, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

Re: Not able to run single command over the notebook

AnmolGan81 — Thu, 09 Jan 2025 12:19:44 GMT

Yeah I figured out the issue yesterday, thanks for all the help.