Handling Traffic Spikes Through Automated Prioritized Load Shedding
summary
Netflix builds an automated per-cluster system that generates, experiments on, validates, and deploys prioritized load-shedding and autoscaling configs, claiming it lets them run fewer and smaller clusters while protecting critical calls. It measures per-RPC latency distributions, CPU and queuing, uses Spectator/Atlas for percentile bucketing, and implements decisions with gRPC interceptors and Envoy filters. Experiments route synthetic spikes, observe brief throttling, spin up three servers, and within two minutes restore full success — the rollout is automated and operationally effective.
AI-written summary · about
chapters (9)
- 0:00Traffic spike solutions
- 1:23Why autoscaling alone isn't enough
- 3:51Congestive failure in systems
- 5:40Failure buffers with stoplights
- 7:47Prioritizing critical requests
- 9:32Sharding clusters
- 11:55Automating resilience config
- 14:21Load shedding implementation
- 15:33Q&A
related talks
44mTechniques Netflix Uses to Weather Significant Demand Shifts
20mBetter Resilience Through Self Tuning
42mPredictive Load-Balancing: Unfair but Faster & more Robust
35mSystems Scale Summer Q&A Week 1
47mLoad Shedding—Approaches, Principles, Experiences, and Impact in Service Management
38mChaos Engineering: A Step Towards Resilience
27mBehind the Scenes: Running Critical Services @ Netflix
25mAvoiding Overload in Distributed Systems