Kernel Samepage Merging (KSM) at Meta and Future Improvements to KSM
summary
KSM is used at Meta to deduplicate millions of anonymous pages and saved ~6 GB on a 64 GB node (up to ~5 million pages in stress tests). Installation uses a new process control flag and systemd’s memory KSM, with tracing and /sys/kernel/mm/ksm counters added. Tuning pages to scan from 100 to 4–5k reduces missed sharing; smart scan skipping cuts scanned pages 10–20%, and an auto-tune advisor yields ~20–30% KSMD CPU savings.
AI-written summary · about
chapters (25)
- 0:00Introduction to KSM
- 1:24What is Kernel Samepage Merging
- 2:11How KSM works internally
- 4:20Enabling KSM for applications
- 6:51System-wide KSM configuration
- 8:16Monitoring KSM operations
- 11:01KSM at Meta and Instagram
- 13:32Configuring KSM for production
- 15:00KSM metrics and memory savings
- 17:09Optimizing KSM scans with skip pages
- 18:54Auto-tuning KSM for efficiency
- 23:18Evaluating new workloads for KSM
- 26:53Advanced evaluation techniques
- 29:50Limitations and security concerns
- 31:00Recap of KSM features and improvements
- 32:28Q&A: KSM and DAMON integration
- 33:51Q&A: Auto-tuning based on memory pressure
- 34:54Q&A: KSM documentation
- 35:36Q&A: Auto-scaling implementation
- 36:19Q&A: KSM overhead and performance impact
- 37:47Q&A: Service-level performance changes
- 38:49Q&A: Kernel vs user space deduplication
- 40:15Q&A: KSM on large NUMA systems
- 41:40Q&A: KSM for hardware validation
- 42:06Q&A: KSM and memory accounting
related talks
17mPara-ksm: Parallelized Memory Deduplication with Data Streaming Accelerator
19mKdump memory usage
27mDay 2 Keynote Sessions
22mPreserving guest memory across kexec
27mMitigating speculative execution attacks with ASI - follow up
43mLive in a world with multiple memory types
37mMeta’s CXL Journey and Learnings in Linux
42mDAMON Recipes: Ways to Save Memory Using a Linux Kernel Subsystem in the Real World