Complexity Thinking, Cynefin & Why Testing Metrics All Suck
summary
Argues most testing efforts produce paperwork not insight and that quality is a complex property so metrics like test case counts or 80% automation mislead. Applies Dave Snowden’s Cynefin model to show obvious work suits automation, complicated work needs analysis, and most testing problems live in unknown unknowns requiring parallel experiments, exploratory testing and user studies. Calls out idiotic metrics (bugs per tester, percent automation) and recommends using professional judgment and experiment-driven detection as practical alternatives.
AI-written summary · about
chapters (29)
- 0:00Intro
- 2:05We test
- 2:20Test cases
- 6:42Complex vs Complex
- 8:05Kinetic Model
- 9:25Ordered Linear Systems
- 12:19Application in Testing
- 12:39Simple Problems
- 13:53Complex Problems
- 14:45Complexity in Testing
- 16:37Parallel Experiments
- 17:04Chaos
- 18:51Metrics
- 20:23The Channel Problem
- 21:48Defining Goals
- 22:20Metrics are bothersome
- 24:29A couple of useless metrics
- 27:33Ask a question
- 27:46Most idiotic real life testing metrics
- 28:50Years of experience to judge expertise
- 30:06Why fight
- 30:24Discussion
- 30:36Satanism
- 30:52Why Metrics All Suck
- 34:43Call to Action
- 35:33Complexity
- 35:54Stupidity
- 36:48Conclusion
- 37:18Good enough
related talks
44mThe Test Automation Pyramid and How It Ruins Your Life
34mesting in modern times: A story about quality and value
28mYour Testing Strategy is broken - lets fix it!
29mYour Metrics Are Lying: How to Stop Reporting and Start Learning
20mEffective Testing Approaches for your Application with Cecelia Martinez
16mChallenges of Testing at Scale | Experiments and Experiences
47mTangible Software Quality
15mHow to NOT waste time