Whodunit? A performance murder mystery
summary
A Rust JSON transformer becomes up to 45 times slower after added dedup, split and sort. Benchmarks with Criterion and comparisons expose quadratic dedup as the culprit; git bisect finds the commit; flame graphs show dedup dominated CPU time and deserialization later accounts for ~50% of CPU. Fixes: replace O(n^2) contains-on-vector with a hashmap inserted from the back, use &str slices to avoid heavy String allocations, and micro-optimize splitting; the hashmap-from-back version wins on large-duplicate workloads while the no-dedup baseline still wins for all-unique inputs.
AI-written summary · about
chapters (15)
- 0:00Start
- 3:54It was a dark and stormy night...
- 5:36The data!
- 9:38benchmark (n.)
- 13:16Git bisect
- 13:37Binary search
- 20:36Flame graphs
- 20:46Let's talk about ... this guy
- 21:58Mommy, what's a Elame graph?
- 24:06Other options
- 28:38Deduplication
- 33:54Creating windows
- 35:03How long is a piece of string?
- 36:41Back to the flame graph
- 38:30Final results
related talks
31mBenchmarking and Optimization of Rust Libraries
37mCache is King: Get the Most Bang for Your Buck From Ruby
23mFast by default: algorithmic performance optimization in practice
14mPragmatic performance patterns
33mReaching Ludicrous Speed
27mHigh Performance Architecture
26mAllocating Less: Really Thin Rust Cloud Apps
35mBeyond Spark MLlib: Deduplicating Common Crawl at Scale