Resillient Microservices without the Chaos
summary
Claims that service-level fault injection testing turns existing end-to-end functional tests into exhaustive failure explorations, using static analysis, test synthesis and a Filibuster tool that injects RPC/HTTP and gRPC faults via OpenTelemetry and a JUnit/CLI workflow. Prototype reproduces real bugs (Expedia, MailChimp, Audible, Netflix examples), reports counterexamples for replay, and uses dynamic reduction to cut exhaustive test cost (observed ~75% reduction in experiments). Implementation is usable in local and staging pipelines and surfaces concrete assertion failures to fix.
AI-written summary · about
chapters (27)
- 0:00Intro
- 0:32Microservices Architecture
- 2:40Microservice Architecture: Netflix
- 3:57Microservices: Socio-Technical Problem
- 4:50Challenge #1: Code Quality
- 6:40Challenge #2: Distributed Systems
- 8:33Microservice Fault Tolerance
- 9:52Fault Tolerance: Example
- 10:15What should happen?
- 11:17What actually happens?
- 13:49How Can We Do Better?
- 14:42Motivating Example
- 15:22Incident #1: Bad Deployment
- 16:12Incident #2: 3rd Party Dependency Failure
- 17:04Verifying The (Missing) Test Oracle
- 17:42Service-level Fault Injection Testing
- 19:03Test Adaptation: Failure-Driven Resilience
- 20:16Fault Permutations: What Do We Test?
- 22:14Dynamic Reduction Technique
- 23:51Evaluation
- 26:16Integrating the Approach
- 28:23Automatic Instrumentation
- 29:41JUnit Integration: Example
- 30:50JUnit Integration: Faults
- 32:22JUnit Integration: Conditional Assertions
- 33:24Standalone Filibuster
- 35:16Conclusion
related talks
40mLet it fail: resilience patterns for microservices
34mChaos Debugging: Finding and Fixing Microservice Weak Points
25mResilience testing in modernization projects
42mMicroservice Testing Techniques
01h 01mBreaking your REST APIs to make them stronger!!
26mHow to ensure Testing Robustness in Microservice
47mOn-stage hacking: build 12-factor microservices
40mMeasuring Micro-services