Service Mesh in Kubernetes: It’s Not That Easy
summary
Lyft replaces sidecar polling with an xDS streaming control plane that merges legacy Dynamo-backed IP lists and Kubernetes endpoints, enabling faster convergence but exposing issues: Envoy’s 50% panic routing threshold and five-consecutive-error outlier detection cause brief panic during rapid scale‑ups. Fixes: enable “exclude new hosts from panic calculation,” add a draining XDS state, patch Envoy to respect out‑of‑order health vs discovery, and switch control plane from Endpoints to Pods using an indexer-by-service to send draining updates—reducing 503s but requiring Envoy and Kubernetes patches.
AI-written summary · about
chapters (13)
- 0:00Start
- 3:30Initial Design
- 6:51Panic Routing
- 7:44Scale Up Fix: Envoy Upstream
- 8:20Application & Envoy Startup
- 8:57Kubernetes & Sidecars
- 10:41Summary - Scale Up
- 12:03Undead Hosts
- 12:37Scale Down Issues
- 17:12Kubernetes Endpoints
- 18:07Kubernetes Indexer for Pods
- 19:12Summary - Scale Down
- 20:23Future Work
related talks
28mDeploying hybrid topologies with k8s & Envoy: a look at service discovery
05mService Mesh Architectures Explained - Sidecar and Beyond
22mUnlock Kubernetes & Envoy: Master Your API Ecosystem!
25mApplication Networking on Kubernetes, where are we now?
35mBack To the Basics - Service Mesh Fundamentals
34mService Mesh for Developers
53mDo you need a Service Mesh?
53mService Mesh: What, Why, And How?