How Not to Do Benchmarking
summary
Shows common mistakes when benchmarking JVM code and why modern JVM optimizations can make naïve measurements misleading. Uses negative examples to demonstrate what not to do and teaches correct benchmarking practices, mentioning tools such as JMH and focusing on pitfalls and how to avoid them.
AI-written summary · about
chapters (19)
- 0:00Benchmarking mistakes
- 3:13JVM optimization pitfalls
- 4:37Failure as learning
- 7:49Bad benchmark impact
- 9:13Benchmark types overview
- 10:59JMH for reliable benchmarks
- 14:55Trust only numbers
- 16:45Volatile vs. LazySet speed
- 20:41Simulate workload with JMH
- 24:34Black Hole CPU explained
- 28:49Modern CPU underestimation
- 33:08Cost model flaws
- 36:37Performance engineering gap
- 38:09Measure always, guess never
- 39:07Benchmark forking and hardware
- 40:35Benchmark production code
- 43:45Regression tests and noise
- 46:21Statistics and confidence
- 47:11Production benchmarking challenges
related talks
02h 41mMicro Benchmarking - The Art of Realizing One is Wrong
02h 44mMeasure, don’t guess: benchmarking stories from the trenches
38mBenchmarking Code Performance with JMH
38mYour benchmark may not guide a real application performance
06mIGNITE: How to benchmark … poorly
54mPerformance tests and other lies
35mBenchmarking: You're Doing It Wrong
45mHow difficult can it be to write efficient code?