Building Low Latency Topologies with Envoy
summary
Explains how Envoy primitives are composed to keep traffic local while enabling safe failover: use priority load balancing to prefer same region/zone, let locality weighted selection spill over as availability drops (Envoy uses a default 40% overprovisioning factor), add cross-region retries and degraded health states to shift traffic without full unhealthiness. Describes Istio and Square implementations, production rollout success, and gotchas: TLS connection setup spikes on failover, uneven endpoint distribution, and Kubernetes removing endpoints can prevent expected spillover unless health checks are tuned.
AI-written summary · about
chapters (9)
- 0:00Multi-region load balancing
- 1:05Problem space and requirements
- 2:51Envoy features fell short
- 5:27Priority load balancing
- 10:01Google East/West case study
- 13:13Square topology design
- 17:58Degraded endpoints handling
- 19:01Gotchas in low-latency topologies
- 24:20Summary
related talks
28mDeploying hybrid topologies with k8s & Envoy: a look at service discovery
15mFail Open, Fail Fast: Improving Envoy Resilience in Latency-Critical Systems
37mEnvoy Internals Deep Dive
38mEnvoy Deep Dive
34mIntro: Envoy
33mRunning Envoy as an Edge Proxy
23mDemystifying Load Balancing in Istio
29mgRPC load balancing and Service Mesh