Journey into the World of Chaos Engineering
summary
Carrefour practices Chaos Engineering on Kubernetes using simple pod killers and a richer experiment platform: KaosCube to kill pods randomly and validate autoscaling and PodDisruptionBudgets, and Litmus to run custom scenarios across nodes, network, cloud and applications. They moved from ad hoc tests to scheduled Night of Chaos exercises, rehearse full cluster rebuilds (recovering production in about four hours), and report reduced MTTR on Black Friday from 6 hours to 2 hours while still addressing gaps in stateful backups.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:17Chaos engineering origins
- 2:46Chaos Monkey history
- 4:52Chaos vs vaccination
- 6:18SLA and MTTR metrics
- 7:00Carrefour cloud context
- 9:31Microservices issues
- 10:14KaosCube benefits
- 13:50MTTR reduction with KaosCube
- 15:1331 October incident impact
- 16:46Litmus Chaos role
- 19:17Litmus Chaos pros and cons
- 21:25Night of Chaos intro
- 25:01Night of Chaos results
- 27:59Accidental operator deletion
- 29:02Future projects
- 30:07Q&A
- 32:59Metrics automation
- 35:28Database chaos
- 38:21Conclusion
related talks
39mChaos: Breaking your Systems to Make them Unbreakable
45mThe Glorious Art of Destroying Things on Purpose: an Introduction to Chaos Engineering
32mImplementing Chaos Engineering in Kubernetes
29mCase Study: Bringing Chaos Engineering to the Cloud Native Develo...
39mResilience & Chaos Engineering in the Cloud
46mDesigning a System for Chaos
19mChaos Engineering at Scale
30mBringing Order to Chaos: Make Your Systems More Resilient with Chaos Engineering