Your Attention, Please: Optimizing Metrics for Humans
summary
Argues that monitoring must optimize for human attention by measuring a system’s demand/output/efficiency/capacity rather than raw CPU/memory/disk; replaces arbitrary thresholds with quantile shape-based alerts that detect when the resource–demand relationship breaks. Gives concrete deployments (a 1,200‑pod Kubernetes cluster, a concert ticketing site, a 1,000‑patient EMR) and reports it works: 98% reduction in false positives and ~1500% improvement in predictive correlation, with firms expanding and backporting the approach.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 3:54Why measure
- 5:05Humans are responsible
- 6:03Our predisposition to simple solutions
- 6:59Lightning storms
- 8:21Simple explanations
- 10:02Optimizing timber yield
- 13:39Measuring everything
- 15:23Sebastian de Vouban
- 19:50sabotaging
- 22:48information
- 23:42real systems
- 26:30embrace complexity
- 27:17system metrics
- 27:45demand
- 28:22output
- 30:52understanding the system
- 31:46the tradeoff
- 32:49concrete examples
- 36:32accidental complexity
- 39:40quantiles
- 43:18thresholdbased alerting
- 44:46other metrics
- 45:56optimization for attention
- 46:14is it worth doing this
- 48:13Takeaways
related talks
18mBad Metrics and Very Bad Decisions
04mWe need a bigger goal than collecting data
37mUX Design and Education for Effective Monitoring Tools
34mMetrics driven development
26mMetrics and Modeling – Helping Teams See How to Improve
48mStanding Up for the Unmeasurable in The Design Process
43mLooking to social science for help with metrics
30mHow to collect application metrics if it needs to be done yesterday