Preventing disaster with data testing: Real-life examples and best practices
summary
Three real failures motivate practical fixes: Mars Climate Orbiter: $327.2M loss from unit mismatch, a flawed 2013 study that wrongly claims ~400,000 deaths/year, and Zillow: $38M lost and 2,000 layoffs from an overconfident pricing algorithm. Recommendations are concrete: use tdda for reference tests and diffs, hypothesis for property‑based checks, Great Expectations and dbt for schema and pipeline assertions, run Monte Carlo and replication experiments, pair‑review analyses, and treat test‑driven data work as modular assertions rather than classic TDD.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:50Real-life data disasters
- 7:11Software to data testing
- 15:26Testing data dimensions
- 19:24Data testing tools
- 23:18TDDA library demo
- 24:21Testing trade-offs
related talks
34mBalancing Early Testing with Real-World Reliability
42mTesting on Thin Ice: Chipping Away at Test Unpredictability
28mTesting Isn't Enough: Fighting Bugs with Hacks
56mTesting at scale
47mTesting on Thin Ice: Chipping Away at Test Unpredictability
15mWhy I Prefer Good Testing Over Excellent Testing
20mShift-left testing : Building reliable Data Pipelines
42mTesting a Data Science Model