Better Resilience Through Self Tuning
summary
Netflix-style control plane uses adaptive concurrency limits, hedge requests, and choice-of-two load balancing to keep services responsive. The system handles ~50k RPS (≈3M requests/min), turns away overload instead of failing, and masks slow backends caused by GC pauses so clients see ~1 ms median and ~80 ms p99 with 99.999% availability. Simulations and production metrics show fewer than 1 in 100k requests fail and overall tail latency is dramatically reduced.
AI-written summary · about
chapters (7)
- 0:00Self-tuning resilience intro
- 1:46Tail latency challenges
- 4:12Concurrency limiters
- 11:34Hedge requests
- 14:03Load balancing bad servers
- 15:27Putting it all together at Netflix
- 18:18Latency and availability results
related talks
44mTechniques Netflix Uses to Weather Significant Demand Shifts
21mHandling Traffic Spikes Through Automated Prioritized Load Shedding
38mChaos Engineering: A Step Towards Resilience
37mReal-Time Adaptive Controls for Resilient Distributed Systems
27mBehind the Scenes: Running Critical Services @ Netflix
18mReal-time adaptive controls for resilient container workloads
19mKeynote: Making and Keeping Netflix Highly Available
51mPatterns for building resilient software systems