HALP: Heuristic Aided Learned Preference Eviction Policy for YouTube Content Delivery...
summary
HALP runs as YouTube’s DRAM eviction layer since early 2022 and claims a 9.1% average reduction in P95 byte-miss during peak with ~1.8% CPU overhead. It filters RRU tail entries (empirically 4 eviction candidates), ranks them with a lightweight two‑layer MLP using pairwise comparisons (prediction ≈720 ns) and trains online from saved comparison snapshots. For measurement it adds a no‑op group and fits noise via impact distribution analysis (MLE) to denoise A/B tests and report robust per-rack effects.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:28CDN Cache Levels: DRAM, SSD, HDD, Origin
- 1:35Three Challenges for Learned Eviction Algorithm Deployment
- 2:07ML Computation Overhead
- 3:09Reducing Avg BMR Without Making Any Location Worse
- 3:36Measuring New Alg Impact Under Production Noise
- 6:12Heuristic Aided Learned Preference (HALP)
- 7:36HALP Design Details
- 10:12Model & Features
- 11:17Recover Impact Distribution from Measurement and Noise
- 12:11Evaluation Setup
- 13:51HALP Has a Modest CPU Overhead of 1.8%
- 15:15Conclusion
related talks
23mML-Powered Search & Recommendation 101
15m3L-Cache: Low Overhead and Precise Learning-based Eviction Policy for Caches
17mPredict, Prune, Play: Efficient Video Playback Optimization Under Device...
24mImproving YouTube Recommender systems for big screen users
23mMachine Learning for ABR in production
49mScheduling with Time-Evolving Uncertainty for Content Review Prioritization in Social Media
20mLearning Relaxed Belady for Content Distribution Network Caching
15mML Algorithms for Trust and Safety