Chaos: Breaking your Systems to Make them Unbreakable
summary
Chaos engineering runs planned experiments on staging then production, using game days last 90 minutes with 30 minutes planning, 50 minutes play and 10 minutes reporting. Tests range from deleting a pod or deployment to blocking dependencies, terminating hosts, degrading CPU/memory, spiking traffic and even killing a region. Practical tools include stress-ng, Vegeta, iptables and simple systemctl commands; vendors like Gremlin and Chaos Toolkit are optional. A WordPress+MySQL demo on Kubernetes shows pods auto-restart, deleting a deployment yields 500 errors, and maxing memory unexpectedly kills the kube master — fix with pod priorities.
AI-written summary · about
chapters (14)
- 0:00Start
- 2:39Difference between Testing and Production
- 8:38Chaos Monkey
- 10:28Game Days
- 10:44Set Time Limits
- 12:05Form a Hypothesis
- 12:31Write Down What You Expect To Do if Everything Goes Horribly Wrong
- 16:16Monitor for Outages
- 20:31Terminate the Host
- 21:09Degrade the Environment
- 24:55Kubernetes
- 27:11Kill My Sequel
- 28:15Degrading the Environment
- 38:09Additional Resources
related talks
46mDesigning a System for Chaos
37mChaos Engineering: The Fine Art of Breaking Stuff in Production on Purpose
01h 02mPractical Chaos Engineering breaking things on purpose to make them more resilient against failure
45mThe Glorious Art of Destroying Things on Purpose: an Introduction to Chaos Engineering
40mJourney into the World of Chaos Engineering
57mChaos Engineering Primer
25mDestroy Your System To Make It More Reliable: An Easy Way To Start Chaos Engineering With LitmusChao
18mChaos Engineering: Break It On Purpose