Perception and bias and metrics, oh my!
summary
Claims that everyone shows bias and that a hidden bias blind spot skews metric interpretation, backed by the fact that there are 180+ biases catalogued; gives concrete examples — anchoring on a 3s page load so a 6s load still feels slow, averages hiding wildly different distributions, and vanity metrics like downloads or pageviews that flatter but mislead. Recommends concrete practices: visualize distributions not means, set achievable anchors from your own data, and build a culture that welcomes questions rather than shames them.
AI-written summary · about
chapters (7)
- 0:00Intro
- 1:00Why do biases exist
- 1:37Why we deal with biases
- 2:16Vanity metrics
- 2:47Narrative bias
- 3:17Visualize
- 4:02Conclusion
related talks
32mIntroducing Reality into Data Metrics
31mFlagging *and* Fixing Bias in ML
14mWhat dashboards don't tell you
43mLooking to social science for help with metrics
18mBad Metrics and Very Bad Decisions
26mMetrics and Modeling – Helping Teams See How to Improve
27mThe Subtle Art of Lying with Statistics
43mThe Subtle Science of Misleading with Statistics