Eliminating bugs: testing is not enough
summary
Claims that testing alone fails because tests don't scale and error propagation loses information; concrete bug: a SIP-style address with a port >65535 travels through Python 2.7 as a plain int, raises an unhandled exception and crashes the UI. Fix implemented as a factory that parses strings into a strong refined/optional type (Optional/Maybe), added type hints and CI checks, shipped the patch next week and the customer stays with no recurrence for six years.
AI-written summary · about
chapters (19)
- 0:00Talk introduction
- 1:07Customer crisis reveals bug
- 6:40Testing alone insufficient
- 7:04Testing doesn’t scale, errors propagate
- 10:08Embed tests as contracts
- 13:22Error propagation: in-band vs exceptions
- 17:23Impossible states unrepresentable
- 21:45Total vs partial functions impact
- 22:59Strong types enforce constraints
- 27:22Optional types for errors
- 30:11Factory methods validate domain
- 33:48Codeify constraints with templates
- 36:20Paths don’t validate: convert
- 37:02Refined types compile-time checks
- 40:57Dependent types beyond constraints
- 45:11Avoid primitive obsession
- 47:19Tighter constraints reduce testing
- 49:29Natural vs formal specs in AI
- 51:59Closing thoughts
related talks
18mHunting the Bugs
43mAutomated Tests Are Not Enough
32mMore testing with fewer tests
35m82 Bugs I Collected in a Year You Won’t Believe Made It to Production
28mTesting Isn't Enough: Fighting Bugs with Hacks
52mFantastic C++ Bugs and How to Test Them
43mOmg, how do I write software that isn't a ticking timebomb!?
36mHow to defuse a bomb Wait, I mean a bug