"Test Repeatability"
summary
Claim: variability trumps repeatability for finding new, important bugs. Evidence: run the same checks with different inputs, different platforms (even “hundreds” a week), different environments, and alternate UI vs API paths instead of always replaying the same script. Practical tactics: change order of operations, vary test data and navigation methods. Repeatable tests still serve regression confidence in CI, but deliberate, relevant variation uncovers risks that sameness misses.
AI-written summary · about
related talks
08mVariations During Testing
35mEffective Debugging of Non-Reproducible Bugs
43mRegression Obsession
45mImproved Regression Testing in Agile
46mThe Road to Reliable Automated Tests
08mStop the Retries in Tests & Reruns of Failing Test
30mA Partial Taxonomy of Test Unreliability
41mDeterministic Solutions to Intermittent Failures