The First 80% of Reading One Billion Rows Fast Enough
summary
Baseline naive Streams run about six minutes on the supplied 13–14 GB CSV while the contest winner gets down to ~1.2 seconds; this presentation shows the 1BRC One Billion Row Challenge path to the first 80% using Java 21 and plain code, cutting allocations, replacing split/double parsing/hashmap with byte-buffer scanning, int math and an open-address table to reach ~30–50 seconds on the author’s machines, drop heap churn to a few megabytes and become effectively async-profiler visible and GC-less by eliminating object churn.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 4:47Challenge rules and setup
- 9:07Baseline with Java Streams
- 12:14Profiling the baseline
- 12:58Classic Java loops
- 15:04Optimizing string splitting
- 17:17Custom integer parsing
- 20:31No intermediate strings
- 21:55Integer temperatures
- 24:07Mutable objects
- 27:23Custom hash set
- 30:38Defer string processing
- 33:53Epsilon GC
- 37:05MinMax optimizations
- 39:53Final 20% speedup
- 44:13Key takeaways and closing remarks
related talks
47mConquering a Billion Rows with Modern Java Concurrency
49m1BRC – Nerd Sniping the Java Community
40mHow Fast Can You Parse 1 Billion Rows in Java? – Insane Speed Test
46mLessons from 1BRC: Tricks and Approaches to Performance Optimization
02h 57mPushing Java to the Limits: Processing a Billion Rows in under 2 Seconds
42mJava, How Fast Can You Parse 1 Billion Rows of Weather Data?
47mUltralow Latency with Java 11 and Terabytes of Data
51mSecrets of top performance in Java revealed