Failing forward to 1 million requests per second
summary
Five consecutive deploys fail during a migration to Envoy while trying to hit 1 million requests per second claims; testing uses wrk2 against a single-host cluster and a configurable no-op upstream. Root causes found and fixed: Envoy’s default 1000 outstanding-request circuit breaker, load-balancer connection buffering (1 MB → 32 KB), docker bridge vs loopback networking, and uneven worker affinity resolved with reuse_port. After fixes throughput climbs from ~15k to ~60k rps and regional failover works.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:11Background
- 1:41Envoy
- 2:22Test setup
- 3:35Results
- 4:28Lets go
- 4:38Envoys circuit breaker
- 6:02Graph
- 8:41Scaling
- 9:16Abyss
- 9:37The Abyss
- 12:09Thread Pools
- 12:46Envoy Reuse Port
- 13:38Test Results
- 14:15End of Journey
- 15:28Suggestions
- 17:54Questions
- 24:30Filters
related talks
22mThe Trials and Success of Adopting Envoy at Tinder
09mLightning Talk: Honey, I Broke the Things: Debugging Gray Failures in Production!
13mHoney, I Broke the Things: Debugging Gray Failures...
39mNetwork Observability with Envoy
24mUnraveling Istio: A Tale of Scale, Challenges and Triumphs at DevRev
01h 32mHands-on Workshop: Envoy Metrics Demystified Workshop
33mEnvoy on Fire: A Practical Look at Debugging a Service Mesh
26mGrowing Pain: Scaling to 10K Workflows per Week