The Future of Scala Performance on Big Data
summary
A Scala big‑data benchmark suite extracts hot spots from real workloads TCPx-BB, SparkBench, GATK/ADAM and reproduces those bottlenecks on a laptop using warm‑up, dynamic input scaling and Java Flight Recorder output so it runs in minutes not hours. The team prototypes fixes — notably replacing degenerate mutable HashMap paths with opportunistic red‑black trees to avoid O(1)→O(n) collapse — and reports measurable speedups in ADAM genomics.
AI-written summary · about
chapters (11)
- 0:00Start
- 0:10Intel and Scala?
- 1:29No agenda no meeting?
- 4:54Benchmark Suite Overview
- 8:43Benchmark Creation Process
- 9:15Some Perf Bottlenecks Found
- 10:03Running the Suite
- 13:17Example Output
- 14:10Example of Success
- 15:44Example Benchmark
- 17:00Conclusions
related talks
38mCharacterizing the performance of Big Data processing platforms
49mNumbers every Scala programmer should know
40mPerformance Testing and Big Data
50mIs scala code slow?
01h 06mBest practices when accessing Big Data or any other data!
35mFast startup & low latency: pick two
39mMemory Efficient Java
48mFantastic Performance and Where to Find It