Intelligent Resource Scheduling for Co-located Latency-critical Services: A Multi-Model
summary
Claims an ML scheduler, OSML, that detects resource cliffs and finds the optimal allocation area using three collaborative models (Model A predicts cliffs/OA, Model B balances QoS vs resources, Model C is reinforcement learning for recovery). It trains on a dataset from 11 services with 62 million allocation cases and ~2 billion samples, runs on Intel Xeon E5 (36 logical cores, 45 MB LLC) using Intel CAT/MBA, and in experiments meets 99th‑percentile QoS faster than baselines, reduces convergence time, supports higher load, and in one case saves three cores and nine LLC ways compared with other schedulers.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:27Executive Summary
- 1:35Monolithic Architecture
- 3:07Key Observations: RCliff and OAA
- 4:20Issues the Existing Schedulers May Meet
- 6:16OSML - Using ML for Resource Scheduling
- 6:52Summary of the ML Models
- 7:54Model-B: Balancing QoS and Resources
- 8:35Model-C: Handling the Changes On the Fly
- 9:20Central Logic - Using ML Models in a Pipelined Way
- 12:06Performance Distribution
- 12:53Resource Usage Comparisons
- 13:58Performance for Workload Churn
- 14:41Generalization
related talks
47mImprove Load Balancing With Machine Learning Techniques Based on the Sched_ext Framework
19mResource Management Innovation for AI Workloads in Cloud Computing
15mUSHER: Holistic Interference Avoidance for Resource Optimized ML Inference
05mScheduling at the Edge of Reason: Multi-Cluster AI & OCM
16mCASSINI: Network-Aware Job Scheduling in Machine Learning Clusters
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
16mMAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale
12mElastic Resource Sharing for Distributed Deep Learning