Industrial Strength Testing for Complex Software
summary
Seq is a self-hosted diagnostic server with a Rust DB engine (now larger than its C# UI) built by a four-person team and in production for over 13 years in production. Testing combines property testing (QuickCheck-style), a query-optimizer fuzzer that generates constrained random queries, instrumented coverage-guided fuzzing using cargo-fuzz/AFL, Miri and fault-injection for IO, and a column-store harness that enumerates encodings over 3,000 rows. These tools find real issues (a Unicode string-equality regression lived five years), produce repros, and tighten long-lived releases.
AI-written summary · about
chapters (13)
- 0:00Industrial testing intro
- 1:31Testing safety-oriented
- 9:54Testing in small teams
- 15:30Software complexity hazards
- 17:37Strategic unit tests
- 24:18Property-based testing
- 28:30Fuzzing and test generation
- 33:23Instrumented fuzzing
- 45:52AFL and advanced fuzzing
- 51:50Model-based validation
- 57:13Fault injection testing
- 1:00:22Dogfooding and burn-in
- 1:02:28Key testing takeaways
related talks
47mTesting the Hard Stuff and Staying Sane
59m.NET Testing Techniques You Didn’t Know You Needed
15mIronSpec: Increasing the Reliability of Formal Specifications
42mSurprise yourself with Property Based Testing
36mFrom Sledgehammer to Fine Brush for QA
38mLiving with Spec
55mNext-Gen Testing Tools for Java
01h 05mMicrotesting