Forum Discussion
Ayush05-gateway
Helper II
Helper IISpark notebook not taking all executors, only running on 1
Dear community, I need some guidance. I'm using the starter pool with 1–10 executors, but when my PySpark notebook runs, it usually starts with only one executor. Sometimes it takes almost a mi...
hi v-prasare I had to re-write my code and break it into modules so I can efficiently uses executors and reducing the no. of rows to be processed.
However this was not the most ideal use of my time.
Hoping to see more flexibility around use of executors in future.
Ayush05-gatewayHelper II
Helper IIhi v-prasare I did submit an Idea, thanks. Marking this topic as resolved. though no solution was used, just re-writing logic.