Using AI, statistics, and set theory to know what to test
summary
Claims target determination from the build graph plus heuristics and ML cut test runs dramatically: tools named include Bazel, NX, TurboRepo, Gradle, Maven, npm, an implemented coverage‑mapping system yields about 50% fewer tests, and Meta‑style LLM ranking reports ~95% recall in their corpora. Details cover how coverage mapping is stored, how transitive deps are inferred, flaky‑test handling, and pros/cons of sharding versus remote execution — concrete methods, measured outcomes, and tooling tradeoffs.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 1:12Continuous integration matters
- 3:42Testing and risk management
- 6:55Target determination with set theory
- 10:06When to use target determination
- 11:30Repository shape affects testing
- 12:53Heuristics to reduce test runs
- 14:00Historical coverage data for selection
- 16:33AI in test selection
- 17:56Predictive test selection with LLMs
- 22:32Strong CI signal importance
- 24:23AI as test coordinator
- 27:31Speed safety cost balance
- 28:56Parallel execution strategies
- 31:44Doing less for efficiency
- 33:11Revisiting test pyramid
- 34:59Smallest possible tests
- 36:02Deleting obsolete tests
- 37:27Reviewing test suites
- 38:08Key takeaways
- 40:17Q&A
related talks
03mTest coverage
44mTest Intelligence: Finding More Bugs in Less Time with AI
25mWhat’s Our Job When The Machines Do Testing?
16mTransforming Test Automation with AI and Machine Learning
08m“Test Recommendation Engine”
26mMaximizing Test Coverage with AI-Powered Test Automation
02h 46mThe Joy of Testing
15m"Linux Testing made better with DATA"