https://community.fabric.microsoft.com/t5/Data-Engineering/Method-to-report-on-node-allocation-usage-in-capacity/m-p/4280649#M5144 <P>Is there an easy way of getting over time the number of nodes allocated for Spark jobs/notebooks?<BR />We can obviously report on individual jobs that are running (Monitor) and the number of CUs in use over time (Capacity Metrics), but one issue we're having is spark jobs grabbing a whole bunch of nodes and then causing the 'not enough nodes' error for other users.&nbsp; I'd like to both monitor node allocations as well maybe trigger alerts when we get close to the limit.<BR /><BR />I'm already having to go on the hunt for rogue users - I'd like to catch it before it happens.</P> Tue, 12 Nov 2024 22:52:30 GMT spencer_sa 2024-11-12T22:52:30Z https://community.fabric.microsoft.com/t5/Data-Engineering/Method-to-report-on-node-allocation-usage-in-capacity/m-p/4280649#M5144 <P>Is there an easy way of getting over time the number of nodes allocated for Spark jobs/notebooks?<BR />We can obviously report on individual jobs that are running (Monitor) and the number of CUs in use over time (Capacity Metrics), but one issue we're having is spark jobs grabbing a whole bunch of nodes and then causing the 'not enough nodes' error for other users.&nbsp; I'd like to both monitor node allocations as well maybe trigger alerts when we get close to the limit.<BR /><BR />I'm already having to go on the hunt for rogue users - I'd like to catch it before it happens.</P> Tue, 12 Nov 2024 22:52:30 GMT https://community.fabric.microsoft.com/t5/Data-Engineering/Method-to-report-on-node-allocation-usage-in-capacity/m-p/4280649#M5144 spencer_sa 2024-11-12T22:52:30Z https://community.fabric.microsoft.com/t5/Data-Engineering/Method-to-report-on-node-allocation-usage-in-capacity/m-p/4280895#M5147 <P>Hi&nbsp;<a href="javascript:void(0)" data-lia-user-mentions="" data-lia-user-uid="679603" data-lia-user-login="spencer_sa" class="lia-mention lia-mention-user">spencer_sa</a>,</P> <P>Perhaps you can try to use real-time intelligence eventsteam with event or alert to notice based on current idle spark nodes.</P> <P><A href="https://learn.microsoft.com/en-us/fabric/real-time-hub/set-alerts-fabric-workspace-item-events" target="_blank">Set alerts on Fabric workspace item events in Real-Time hub - Microsoft Fabric | Microsoft Learn</A></P> <P>Regards,</P> <P>Xiaoxin Sheng</P> Wed, 13 Nov 2024 03:22:32 GMT https://community.fabric.microsoft.com/t5/Data-Engineering/Method-to-report-on-node-allocation-usage-in-capacity/m-p/4280895#M5147 Anonymous 2024-11-13T03:22:32Z