topic Re: Spark starter pool is not working with F4 capacity in Data Engineering

Spark starter pool is not working with F4 capacity

marcoG — Sun, 19 Nov 2023 18:13:40 GMT

Spark starter pool is not working.

I have F4 capacity with 1 medium node.

I have to wait about 3 minutes for it to start instead of a few seconds

Re: Spark starter pool is not working with F4 capacity

Anonymous — Mon, 20 Nov 2023 07:44:37 GMT

Hi @marcoG ,

Apologies for the issue you have been facing.
We are reaching out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Re: Spark starter pool is not working with F4 capacity

Anonymous — Mon, 20 Nov 2023 10:02:11 GMT

This seems like the same issue that was described here: https://community.fabric.microsoft.com/t5/General-Discussion/Slow-starter-pool-session-initialization/td-p/3507465 - I have not seen any of the documents being updated with "known issues" or received any update on this matter yet.

Re: Spark starter pool is not working with F4 capacity

marcoG — Mon, 20 Nov 2023 10:06:48 GMT

It is similar but not the same.

I have F4 capacity not F2.

F4 Capacity it have 8 spark core(1 Medium node) enough to use Spark Starter Pool.

I thing there is some bugs add with enviroment preview

Re: Spark starter pool is not working with F4 capacity

Anonymous — Mon, 20 Nov 2023 11:26:52 GMT

Hi @marcoG ,

I am trying to get the confirmation on the same. We have a bug filed for F2.
Let me check with the internal for F4.

Appreciate your patience.

Re: Spark starter pool is not working with F4 capacity

Anonymous — Tue, 21 Nov 2023 05:44:51 GMT

Hi @marcoG ,

Apologies for the issue you have been facing. This is a known issue as and we are working on addressing the starter pools when the max node configuration is set to 1
have called this our in the known issues section of starter pools documentation: Link

Hope this helps. Please let us know if you have any further issues.

Re: Spark starter pool is not working with F4 capacity

Anonymous — Tue, 21 Nov 2023 09:01:32 GMT

Hi @marcoG ,

Glad to know that your query got resolved. Please continue using Fabric Commuity for your further queries.

Re: Spark starter pool is not working with F4 capacity

marcoG — Thu, 11 Apr 2024 12:53:46 GMT

It seems to me that the problem has been solved