Load Shedding—Approaches, Principles, Experiences, and Impact in Service Management
summary
Google-style load shedding deploys a local, fast verdict system that uses in‑flight per‑request CPU accounting, soft/hard thresholds and tagged traffic to protect tasks from overload, plus a decoupled policy layer so teams can tune behavior. Implementation adds four priority bands, client-side retries with backoff/hedging, and batching tags so batch and production capacity merge. Result: a service moves from ~25% provisioned headroom to ~85% utilization, handles spikes that last 15–20s, cuts tail latency by orders of magnitude, and avoids global outages.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 1:30Why are we here discussing load shedding
- 2:09The 15 moments of fame
- 3:43Threads to a Service
- 5:53Blood Chaining
- 7:15Cascading Failure
- 8:41Sharing Traffic
- 11:16Principles
- 17:40Survive
- 18:26Isolation
- 18:51Request
- 20:16Cost
- 20:53Retries
- 22:19Areas of Focus
- 24:10Typical Application Layout
- 25:05How do you know your task is loaded
- 29:03Task capacity
- 29:30Aggregate capacity
- 30:01Rebalancing traffic
- 31:00Blocking unexpected spikes
- 32:20The parameters
- 32:59Impact
- 36:12Reserve Margin
- 36:25Batch and Production
- 37:55Batch Handling
related talks
21mHandling Traffic Spikes Through Automated Prioritized Load Shedding
25mAvoiding Overload in Distributed Systems
13mWhen to Hedge in Interactive Services
44mTechniques Netflix Uses to Weather Significant Demand Shifts
32mHow does your application behaves when everything goes wrong?
49mHow We Protect Millions of Customers During Overloads
35mSystems Scale Summer Q&A Week 1
13mAccept Partial Failures, Minimize Service Loss