Keynote: Benchee: 9 Years of Benchmarking on the BEAM
summary
Benchee becomes a practical, extensible benchmarking suite: a layered architecture that lets plugins replace collection, formatting and stats; it adds memory and reduction measurements (separate runs per process) and an inputs feature that revealed a real-world bug where a query with 2.3M rows behaves faster than the “no rows” case until a compound index fixes it. Benchmarks show surprising results—body recursion can beat tail recursion at some sizes, flat_map regression gets fixed—and defaults (overhead subtraction) once caused wrong results.
AI-written summary · about
chapters (11)
- 0:00Benchee introduction
- 5:28What is benchmarking
- 12:35Benchee's birth and purpose
- 21:00Designing Benchee
- 31:12Tail vs body recursion
- 35:24flat_map performance
- 36:49Benchee in real apps
- 41:299 years of lessons
- 45:45Open source challenges
- 50:39Benchee's future
- 53:29Q&A
related talks
01h 03mBenchmarking and optimizing the Carbon compiler
40mKeynote: Fireside Chat on code design
29mKeynote: BDD - Bias driven development
01h 19mKeynote: I Can Write the Code But Getting Something Done Is Another Matter
01h 00mMeasure, Then Build
01h 00mClosing Keynote: After a Decade, Still a Rubyist
01h 04mUnlocking Performance Through Reverse Engineering
23mBeating the Benchmarks: Co-evolving the ISA and Development Tools