When Gremlins Play with Cockroaches
summary
Experiment uses Gremlin "Failure as a Service" against CockroachDB serverless then the open-source dedicated build, trying Docker, Minikube and three AWS instances. Serverless blocks Gremlin because no admin API access; Docker shows one node constantly failing; switching to Minikube resolves the node failure. AWS install takes six painful hours and remains unfinished. Outcome is a documented set of failures and workarounds, partial reproducible experiments, and a plan to try Netlify/Heroku and alternative chaos tools next.
AI-written summary · about
chapters (9)
- 0:00Chaos engineering intro
- 1:05Exploring chaos tools
- 3:28Gremlin with CockroachDB
- 5:55Switching to Serverless
- 7:43Docker and Gremlin issues
- 9:32Node failures and Kubernetes
- 10:55AWS for the experiment
- 11:39Lessons learned
- 13:05Next steps
related talks
49mPerforming Chaos in a Serverless World
27mAutomating Resilience Testing with Clojure and Docker
35mChaos Engineering WG Deep Dive
42mUse Docker to test microservices
27mPutting Chaos Into Continuous Delivery to Increase App...
43mDebug Like a Scientist!
25mTesting in production: Ideas, experiences, limits, roadblocks
34mAre You About to Break Prod? Acceptance Testing with Ephemeral Environments