How does Fabric handle concurrent real-time ML inference requests?

HI vkeda29​,
Microsoft Fabric ML model endpoints support XGBoost and LightGBM model flavors, subject to the documented schema limitations. The scoring API accepts multiple input records in a single request. Fabric manages the endpoint infrastructure and scales resource usage based on incoming traffic, up to three nodes per endpoint; however, documented maximum concurrency and guaranteed throughput limits are not specified. Handle HTTP 429 responses using the Retry-After header, and consider disabling auto sleep to avoid wake-up delays after inactivity. Before production deployment, load-test representative batch sizes and concurrent requests, monitoring response times, errors, throughput, and Fabric capacity consumption.
Serve real-time predictions with ML model endpoints (Preview) - Microsoft Fabric | Microsoft Learn
Endpoint - Score ML Model Endpoint - REST API (MLModel) | Microsoft Learn
Throttling - Microsoft Fabric REST APIs | Microsoft Learn
What is the Microsoft Fabric Capacity Metrics app? - Microsoft Fabric | Microsoft Learn

Thank you.