When Life Gives You Lemons, Are You Counting Them or Making Lemonade?
summary
Argues that common KPIs like tests executed or bugs reported mislead decision making and offers a practical 7-step framework to design value-focused indicators: define context, form hypotheses, vet data quality, model, validate, iterate, and communicate. Shows concrete data work—sourcing and sanitizing over 200 football divisions down to ~100 comparable leagues to build a “farmers league” index—and recommends tools and checklists (Ministry of Testing one‑page plan, AI helpers like Copilot/Gemini) to avoid vanity metrics and the garbage in, garbage out trap.
AI-written summary · about
chapters (10)
- 0:00Welcome
- 3:17Vanity metrics dangers
- 9:50Data quality insights
- 12:19Metrics to KPIs steps
- 20:58Data modeling relevance
- 23:26Validating insights
- 26:18Refining KPIs
- 31:17Data storytelling
- 36:16From lemons to lemonade
- 39:56Q&A
related talks
26mMetrics and Modeling – Helping Teams See How to Improve
29mYour Metrics Are Lying: How to Stop Reporting and Start Learning
22mPractical Visualization of the Value of Testing for the Client
28mKPIs to Facilitate 'Fail Fast, Learn and Iterate'
34mesting in modern times: A story about quality and value
41mRethinking Measurement for High-Performing Teams
29mC.A.K.E: An Effective Framework for Identifying Your Community KPIs
03mBoris Wolfson at RIT++ 2012