Misnomers About Microbenchmarks
summary
Argues that zero allocations often has negligible user impact and gives concrete measurements: Go's testing/bench with -benchmem and benchstat, wrk for loaded tests, and debug GC stats. Shows AES-GCM runs ~7× faster on x86 with CPU crypto but in real services AES vs secretbox changes tail latency by only ~0.1–0.2 ms at the 99th percentile; router microbenchmarks with millions of requests show no meaningful user-visible difference. Recommends profile first, benchmark on production hardware, run long and randomize order.
AI-written summary · about
chapters (17)
- 0:00Intro
- 1:19First: what is a "micro" benchmark?
- 12:25The order in which you run your benchmarks can impact your results
- 15:40Utter Garbage
- 17:51GC behaviour is your algorithms' impact a meaningful fraction of the larger program?
- 19:33Quantifying the Impact
- 23:09The Great Go Router Wars
- 27:57What's the relationship between these microbenchmarks & application performance?
- 33:10What about GC pause times?
- 33:273.7ms request latency (99th)
- 34:36Another Look
- 39:45Microbenchmarking Best Practices
- 40:16Profile first
- 40:55Mitigate CPU impact by running your benchmarks longer.
- 43:27Does adopting the algorithm that allocates less, or is faster, help you meet your Service Level Objectives? (SLO)
- 44:17What Should We Care About?
- 44:34Use of composable interfaces.
related talks
56mWhy Your Go Benchmarks Are Lying
39mMicro Benchmarks
38mYour benchmark may not guide a real application performance
05mLightning Talk: Micro C++ Benchmarks & Murphy's Law
59mCounting Nanoseconds Microbenchmarking C++ Code
42mPerformance Matters
29mPractical guide to benchmarking your optimizations
39mMemory Efficient Java