Author: Santhosh Kumar Ravindran, Principal Product Manager
Custom Live Pools for Fabric Data Engineering is a new way to schedule and manage “ready-to-run” Spark capacity so your teams spend less time waiting for compute and more time delivering outcomes. Custom Live Pools are deeply integrated with Environment artifacts, giving workspace and capacity admins a clear, manageable unit for compute configuration, libraries, and pool lifecycle management.
The challenge
In real-world data engineering, demand is rarely flat: morning ETL waves, scheduled notebook runs, and peak interactive usage can create bursts where the biggest pain isn’t raw capacity, it’s time-to-readiness and operational predictability.
What’s new
Capabilities with Custom Live Pools:
- Schedule pool activation windows so Spark clusters are available when you need them (and not when you don’t).
- Monitor pool lifecycle and readiness (hydration status, activation details, and utilization) through Monitoring experiences.
- Integrate with Environment-based library and compute management, including clear handling of library publishing vs. existing pooled sessions.
Library management behavior
Because Custom Live Pools are tied to Environments, library updates follow an intentional model: publishing new libraries won’t automatically refresh already active or idle pooled sessions. Instead, users would have to update their libraries to make sure their next pool warm-up is refreshed with the latest library package updates. This gives admins control and avoids surprising mid-flight changes to live compute.
Monitoring and operational visibility
Custom Live Pools include Monitoring experiences designed to give visibility into the state and behavior of the pool lifecycle — from hydration status to environment handoffs and downstream utilization signals through the Monitoring Hub. The activation workflow also includes explicit start / status / cancel scenarios, with activation status intended to support operational tracking.
Networking
Custom Live Pools are intended to work in workspaces with Managed Private Endpoints (MPE) enabled.
Get started
- Navigate to your Environment artifact and configure Custom Live Pool settings (including schedule/activation).
- Publish the Environment so the settings take effect.
- Use Monitoring experiences to validate activation readiness and lifecycle signals.
Next steps
For detailed documentation, refer to the Microsoft Fabric Data Engineering documentation at Configure custom live pools in Microsoft Fabric - Microsoft Fabric | Microsoft Learn We’d love to h... Fabric Community.