Load Shedding for Evil Wizards
summary
Compares Envoy mechanisms with concrete simulations: a Go buffer bloater proxy and an external Envoy produce results showing local rate limiting (token bucket) holds throughput at configured ~2500 RPS but collapses when service time degrades from ~3 ms to ~10 ms; bounding concurrency (circuit breakers) with 8 worker threads keeps good put near 2500; adaptive concurrency measures p95 latency and updates every ~0.25s to track an ideal window and auto-adjust limits; admission control/mission control probabilistically rejects but can oscillate on recovery.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:41What is Load?
- 1:20Little's Law.
- 2:47Little's Law Applied to Servers.
- 5:13Avoiding Collapse
- 6:18Simulating Overload
- 9:08Part II: Examining the Envoy Load Shedding Mechanisms
- 16:54Rate/Bandwidth Limits vs. Concurrency Control
- 17:46Shortcomings of Circuit Breakers
- 18:45What if I told you there was a better way?
- 22:00The gradient.
- 22:32Concurrency Limit Behavior.
- 22:53Configuring Adaptive Concurrency
- 25:35Adaptive Concurrency Drawbacks
- 26:19Admission Control Filter
related talks
43mGoogle - Load-shedding
33mUsing Envoy for data aware traffic routing in Azure Service Fabric
42mStop Rate Limiting! Capacity Management Done Right
26mService Mesh Failure Stories at Scale
51mBuilding Fault Tolerant Microservices
09mLightning Talk: The Hardest Part of Operating a Service Mesh: Envoy Proxy
44mDeep Dive talk | Resilience in Microservices with Congestion Control
25mHow to DDOS yourself with Envoy (and other tales of migration horror)