Improve Load Balancing With Machine Learning Techniques Based on the Sched_ext Framework
summary
An ML-driven, resource-aware load balancer built on sched_ext + eBPF collects scheduling features (task weight, NUMA distance, cache sharing, P/E core type) and ships traces to user space for training and inference, avoiding kernel floating-point changes. The pipeline gathers ~300k decisions in 10 minutes, trains models that reduce task migrations and latency, and for UPF/5G and some compilation workloads outperforms EEVDF, showing notable end-to-end latency and throughput gains while highlighting limits around trace volume and hardware-accelerated inference.
AI-written summary · about
chapters (10)
- 0:00CPU scheduling challenges
- 4:23NUMA and cache topology
- 6:39Heterogeneous computing
- 9:54Latency vs throughput
- 12:07Single-scheduler limits
- 14:57Sched_ext framework
- 17:465G core scheduling case
- 27:08ML for adaptive scheduling
- 33:44Kernel ML challenges
- 37:47Q&A
related talks
36mAccelerating AI training fleets with sched_ext
16mIntelligent Resource Scheduling for Co-located Latency-critical Services: A Multi-Model
14mPerformance Optimization in High-Volume Transaction Systems
46mMachine Learning with Kubernetes
16mKeynote: LLM-Aware Load Balancing in Kubernetes: A New Era of Efficiency
15mUSHER: Holistic Interference Avoidance for Resource Optimized ML Inference
11mScaling Distributed Machine Learning with In-Network Aggregation
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources