Testing The Tests: Mutation Testing for C++
summary
Describes a C++ mutation-testing effort that actually builds mutators (operator swaps, initializer-list vs count ambiguity, removing range-for ampersands, changing lambda capture from value to reference) and runs them with clang/LLVM tooling, using IR-level mutation and LLVM IR mutation for speed plus distributed execution; finds real bugs (crashes from returning refs or capturing locals), flags redundant tests and dead code, and reports practical tradeoffs: equivalent mutants require human filtering, compile-time static_asserts complicate scoring, OpenSSL runs finish in minutes.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:28Why do we test
- 2:31How do we test
- 3:48Testing mutator
- 5:34What can you get from mutation testing
- 9:26Mutations in code
- 13:18Optimizations
- 14:37Testing
- 16:20Design
- 17:46Vectors
- 19:34Other cases
- 21:01Vector names
- 22:10equivalence
- 23:23lambdas
- 25:17lambda equivalence
- 26:48compilation is test
- 27:39Error handling
- 29:45Mutation tools
- 33:41Highlevel mutations
- 36:18How much do you need
related talks
46m"But my tests passed!" - Exploring C++ Test Suite Weaknesses with Mutation Testing
40mMutation Testing: How Good Are Your Tests
20mUnleashing the Power of Mutation Testing: Supercharging Software Quality and Resilien...
35mKill All Mutants! (Intro to Mutation Testing)
01h 06mKill All Mutants! (Intro to Mutation Testing)
50mKill All Mutants! Intro to Mutation Testing
29mKill All Mutants! (Intro to Mutation Testing)
22mKill All Mutants! (Introduction to Mutation Testing)