Using hardware hints for optimal page placement
summary
Kernel consumes AMD EPYC IBS instruction-sampling to get precise physical addresses and data-source (cache/DRAM, local vs remote), driving NUMA balancing and hot‑page promotion. A microbenchmark (64 threads, 8GB remote allocation, 1e9 accesses/thread, 100µs think time) shows IBS can migrate ~8GB and match software scanning on regular nodes but performs more gradually and worse with a CXL remote node. Implementation uses NMI/MSR programming on context switch, taskwork for migration, and tunable sampling 5k–20k µops; replacement is feasible but needs rate tuning and incurs modest overhead.
AI-written summary · about
chapters (15)
- 0:00Introduction to hardware hints
- 1:44Why page access hints matter
- 4:33Hardware-provided access hints
- 7:03Instruction-based sampling details
- 14:33Basics of NUMA balancing
- 19:30Q&A on virtualization and ARM
- 23:03Implementation of IBS-driven NUMA balancing
- 25:55Test setup and benchmarks
- 29:13Results on regular NUMA nodes
- 32:24Results on CXL memory nodes
- 35:34Overhead analysis
- 37:20Virtualization experiment
- 39:49Q&A on cache access and sampling
- 42:41Feedback and future opportunities
- 44:44Q&A and closing remarks
related talks
31mScaling & Optimizing Performance on AMD EPYC Systems
21mPreview: Operating Systems
43mLive in a world with multiple memory types
20mLinux Kernel Scheduling and split-LLC architectures: Overview, Ch...
42mMessing up Your NUMA Topology with CXL
38mMaximizing Heterogeneous Memory Bandwidth in Multi-Socket Systems
43mExpanding the Memory Limits with CXL Software Stack
37mMeta’s CXL Journey and Learnings in Linux