Benchmarketing fallacies: the 100 shades of truth
summary
Examines how benchmark claims break when filters or options remove competing implementations, using teeken power filter that hides ORM usage as a concrete example and showing the published “fastest” line is often apples vs oranges. Demonstrates digging into run logs, running CPU profiles and the USE method to find saturation and context, uses CPU profiling to show core saturation vs apparent DB bottleneck. Explains cognitive tricks—assertive headlines and hindsight bias—that make misleading results stick.
AI-written summary · about
chapters (7)
- 0:00Benchmarking and marketing fallacies
- 7:44Defining benchmarks and bench marketing
- 14:25Spotting broken benchmark claims
- 23:21Psychology of marketing claims
- 29:04Active benchmarking limits
- 35:13Bias, ethics, and truth
- 38:45Q&A
