Fewer, Smarter, Faster: Scaling Testing @Spotify
summary
Spotify runs about 48,000 pre-merge tests and uses Bazel with bazel diff to compute impacted targets so only affected tests run after restructuring features into API, implementation, test utilities and test targets for test isolation. About 65% of components are isolated and PR to green time drops 50–80% while CI startup time falls ~66%. A Master Guardian system detects and skips flaky or slow tests (threshold ~8s) pre-merge and runs all tests post-merge.
AI-written summary · about
chapters (8)
- 0:00Introduction to testing at Spotify
- 1:11Defining scale at Spotify
- 5:01Fewer smarter faster tests approach
- 5:42Test avoidance with Bazel
- 18:54Managing flaky tests
- 21:43Skipping slow tests
- 23:28Key takeaways and recommendations
- 25:03Q&A session
related talks
26mKeeping Your Tests Lean
46mThe state of the Art in Tackling Flaky Tests
10mBeyond “try harder”: Effective strategies to tackle bugs
24mSupercharging CI: Test Suite Optimization for Speed, Stability, and Savings
40mScaling testing & test automation without losing control
17mContinuous Testing at Scale
38mScaling Automated Testing & Architecture from Startup to Enterprise
34mFantastic Bugs and Where to Find Them