Forum Discussion
speed up spark computation
- Anonymous1 year ago
Hi Jeanxyz ,
Thank you for reaching out to us on Microsoft Fabric Community Forum!
Even though you are using SKU2, performance can vary between organizational and personal environments due to differences in how Spark resources are provisioned. Slow package installs like pdfplumber are common if the environment is initializing or the pool is small. To improve this, consider reusing the environment after the first install and increasing the number of nodes if your SKU allows.
Regarding the autoscale settings being greyed out in both the workspace and environment, this happens if the pool is already in use or was created without autoscale enabled. To resolve this, create a new Spark pool with autoscale configured during setup, then assign it to your environment.
Refer the links here:
Stock Keeping Unit (SKU) considerations - Microsoft Fabric | Microsoft Learn
Understand Autoscale compute for Spark detail page - Microsoft Fabric | Microsoft Learn
I hope this resolve your query. If so,give us kudos and consider accepting it as solution.
Regards,
Pallavi.
Hi Jeanxyz ,
Thank you for reaching out to us on Microsoft Fabric Community Forum!
Even though you are using SKU2, performance can vary between organizational and personal environments due to differences in how Spark resources are provisioned. Slow package installs like pdfplumber are common if the environment is initializing or the pool is small. To improve this, consider reusing the environment after the first install and increasing the number of nodes if your SKU allows.
Regarding the autoscale settings being greyed out in both the workspace and environment, this happens if the pool is already in use or was created without autoscale enabled. To resolve this, create a new Spark pool with autoscale configured during setup, then assign it to your environment.
Refer the links here:
Stock Keeping Unit (SKU) considerations - Microsoft Fabric | Microsoft Learn
Understand Autoscale compute for Spark detail page - Microsoft Fabric | Microsoft Learn
I hope this resolve your query. If so,give us kudos and consider accepting it as solution.
Regards,
Pallavi.