Failed to create Livy session for executing notebook.

[ahmetyilmaz]

Hi, I am triggering the pipelines in parallel from a single pipeline, and I get this error. Previously, when the pipelines were running in parallel, they would wait for each other to finish if the capacity was not enough, but now it gives such an error.

Failed to create Livy session for executing notebook. Error: [TooManyRequestsForCapacity] This spark job can't be run because you have hit a spark compute or API rate limit. To run this spark job, cancel an active Spark job through the Monitoring hub, choose a larger capacity SKU, or try again later. HTTP status code: 430 {Learn more} HTTP status code: 430.

[v-prasare]

Hi @ahmetyilmaz,

Thanks for reaching MS Fabric community support.

please refer to below simliar resolved thread from community and let me know if this helps?

 Solved: Error in parallel ForEach - Microsoft Fabric Community

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

[ahmetyilmaz]

Thank you for your answer @v-prasare  . I ran some of the parallel ones in series and that solved the problem. But I think this is a temporary solution.

[v-prasare]

Hi @ahmetyilmaz ,
Thanks for your patienece.

In this scenario i suggest you to raise a support ticket here. so, that they can assit you in addressing the issue you are facing. please follow below link on how to raise a support ticket:

How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

Thanks,

Prashanth

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query