Forum Discussion
Capacity Usage of AI Function Calls
Is the compute consumption of AI function calls delayed in how it registers against capacity usage? Even after the Notebooks stop running during the day, capacity usage continues to rise overnight, despite capacity not reaching 100% (approx 80% reached). Thanks!
Thankyou, lbendlin, for your response.
Hi hp0lkm46,We sincerely appreciate your inquiry posted on the Microsoft Fabric Community Forum.
Based on my understanding, Spark sessions are initiated during the execution of notebooks. Even after the cells have completed execution, these sessions may remain active or idle, thereby continuing to consume capacity unless they are explicitly stopped. Additionally, Fabric employs a compute smoothing mechanism which distributes usage over time to manage sudden spikes. This can sometimes cause the usage to appear delayed or prolonged.
Please follow the steps below, which may help to resolve the issue:
- Stop Spark sessions manually by adding spark.stop() at the end of the notebook to release resources.
- Monitor the status of Spark sessions and check for any sessions still marked as Running or Idle.
- Utilize the Capacity Metrics App to track AI and compute usage trends, and identify ongoing or idle Spark jobs.
Furthermore, kindly refer to the following links for more information:
Understand your Fabric capacity throttling - Microsoft Fabric | Microsoft Learn
Pause and resume your capacity - Microsoft Fabric | Microsoft LearnWe hope the information provided will help to resolve the issue. Should you have any further queries, please feel free to contact the Microsoft Fabric community.
Thank you.
5 Replies
- lbendlin
Super User
After your notebook run has completed you need to force close the spark session.
v-pnaroju-msftCommunity Support
Thankyou, lbendlin, for your response.
Hi hp0lkm46,We sincerely appreciate your inquiry posted on the Microsoft Fabric Community Forum.
Based on my understanding, Spark sessions are initiated during the execution of notebooks. Even after the cells have completed execution, these sessions may remain active or idle, thereby continuing to consume capacity unless they are explicitly stopped. Additionally, Fabric employs a compute smoothing mechanism which distributes usage over time to manage sudden spikes. This can sometimes cause the usage to appear delayed or prolonged.
Please follow the steps below, which may help to resolve the issue:
- Stop Spark sessions manually by adding spark.stop() at the end of the notebook to release resources.
- Monitor the status of Spark sessions and check for any sessions still marked as Running or Idle.
- Utilize the Capacity Metrics App to track AI and compute usage trends, and identify ongoing or idle Spark jobs.
Furthermore, kindly refer to the following links for more information:
Understand your Fabric capacity throttling - Microsoft Fabric | Microsoft Learn
Pause and resume your capacity - Microsoft Fabric | Microsoft LearnWe hope the information provided will help to resolve the issue. Should you have any further queries, please feel free to contact the Microsoft Fabric community.
Thank you.
- v-pnaroju-msft
Hi hp0lkm46,
We would like to follow up and see whether the details we shared have resolved your problem.
If you need any more assistance, please feel free to connect with the Microsoft Fabric community.
Thank you. - v-pnaroju-msft
Hi hp0lkm46,
We are following up to see if what we shared solved your issue. If you need more support, please reach out to the Microsoft Fabric community.
Thank you. - v-pnaroju-msft
Hi hp0lkm46,
We wanted to see if the information we gave helped fix your problem. If you need more help, please feel free to contact the Microsoft Fabric community.
Thank you.
