Replicated service mesh: hardening systems against failure modes in load balancing
summary
Practical recipe for preventing mesh meltdown: prefer L7 load balancing so the proxy can do content routing, traffic splitting and per-request failover; derive cluster capacity from replica CPU and use auto‑scaling plus error‑averse assignment so failing clusters lose effective capacity and traffic shifts away; shed at high utilization (example: ~95%) and use readiness probes to remove slow pods; add circuit breakers, per‑client quotas and tracing (OpenTracing/Jaeger or OpenCensus); enforce limit retries to 10% and prefer externally consistent stores (Spanner/Cockroach).
AI-written summary · about
chapters (18)
- 0:00Introduction and background
- 1:51What is a service mesh
- 3:40Service reliability goals
- 5:48Load balancing in DC
- 10:05Blast radius containment
- 11:32Capacity and failover strategies
- 17:30Understanding overload
- 21:19Preventing overload issues
- 27:20System changes and reliability
- 31:28Data consistency in systems
- 34:35Work conserving and deadlines
- 35:59Observability in service mesh
- 37:45Key takeaways and summary
- 39:15Q&A: Distributed tracing
- 41:48Q&A: Production changes
- 44:39Q&A: Readiness probes
- 46:51Q&A: Secret rotation
- 48:56Q&A: Hard dependencies
related talks
35mDesign patterns for application resiliency
42mDesigning for failure
37mResilience Design Patterns
34mDistributed Systems - What Can Go Wrong and How to Get It Right
25mAvoiding Overload in Distributed Systems
51mPatterns for building resilient software systems
25mSmoke & Mirrors: The Primitives of High Availability
36mDesigning for failure - architecting fault-tolerant system