In the kitchen: A sprinkle of fire and Chaos
summary
Chaos experiments use Gremlin and Kubernetes to run planned failures: observe metrics and SLOs, form a hypothesis, then run a blast radius limited CPU injection (example: ramp CPU to ~60% across a subset of hosts, then increment) while monitoring abort conditions and user-facing errors. Auto-scaling timing, Kubernetes resource limits and third‑party dependency latency are tested; outages cost ~$300,000/hour, so experiments aim to reveal weaknesses, fix them, and verify fixes by replaying the test.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 4:26Cooking and reliability
- 8:36Cost of downtime
- 11:07Why systems grow complex
- 13:55What is chaos engineering
- 17:02Chaos experiment process
- 20:50Practical chaos tests
- 24:42Reliable apps recipe
- 27:50Testing failures
- 28:51Company reliability focus
- 30:15Closing
