Opening the Box: Diagnosing Operating-System Task-Scheduler...
summary
Linux scheduler decisions cause measurable slowdowns on large multicore runs; visualization and extra trace points reveal schedgraph tools (datograph, running-waiting, stepper, events) fed from trace-cmd and trace_printk. A NAS benchmark shows unexpected 31 vs 33 thread imbalance and 33 of 40 failed attempts to find a stealable runqueue because NUMA-move migrations drop CPU concurrency. The custom tools pinpoint the NUMA-move-induced work‑conservation issue and simple kernel tweaks reduce the long-tail slow runs but do not eliminate all variability.
AI-written summary · about
chapters (20)
- 0:00Diagnosing task scheduler
- 2:27Work conservation challenges
- 3:51NUMA locality issues
- 5:14Hard to observe scheduler
- 5:56Trace-cmd for scheduler events
- 8:04Kernel Shark for traces
- 9:50Custom visualization tools
- 11:14Case study: NAS Parallel Benchmark
- 12:58BT benchmark analysis
- 13:40Scheduler utilization gaps
- 15:48Running vs waiting tasks
- 17:54Zooming into migration and load balancing
- 19:16NUMA balancing impact
- 21:43Load balancing failures
- 23:10Active balancing constraints
- 26:18Debugging balance counters
- 28:46Load balancing visibility issues
- 31:54Proposed fixes impact
- 32:15Results after mitigations
- 33:19Conclusion and future work
related talks
21mGraphing tools for scheduler tracing
19mDo nothing fast: How to scale idle cpus?
46mOptimizing Scheduler for Linux Gaming
51mLinux Systems Performance - Architecture
18mScheduler Tracing With ftrace + eBPF
41mLinux Systems Performance
52mEffectively Measure and Reduce Kernel Latencies for Real-time Constraints
26mLatency Distributions and Micro-Benchmarking to Identify and Characterize Kernel Hotspots