Troubleshooting the Performance of a Large Erlang System
summary
Hyper‑Q is a 2 million line Erlang system with ~1000 modules that virtualizes SQL; a customer query returning 16 million rows × 20 columns runs 1 minute on the target DB but Hyper‑Q adds about 4 minutes. Engineers use Cprof/eprof/fprof to triangulate the hotspot to result conversion, implement bespoke formatters (date formatter 16× faster, customer queries ~20% faster), add small C NIFs and binary pattern matching, and cut a 60M‑row run from 300s to 181s while raising conversion from 6.3 to 10.5 MB/s.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:23Introduction to Datometry/Hyper-Q
- 2:08Customer Incident
- 3:12In house reproduction
- 5:41Identifying Dominant Cost
- 7:20Profiling Code
- 13:11Optimization Example
- 16:15Profiling Takeaways
- 17:18Binary Optimizations
- 19:4220% performance improvement
- 21:30Binary Optimization Takeaways
- 22:24Additional optimizations
- 24:49Lessons Learned
related talks
24mGoing CPU Bound
48mVM Tuning, Know Your Engine - Part III: The Scheduler
48mElixir by the Bellyful
40mOpen Riak how we gained from leaning on the BEAM
45mElixir: une potion pour mixer complexité et concurrence en temps réel
53mElixir: a potion to mix complexity and concurrency in real‑time
49mAn Introduction to Erlang From behind the Trenches
03h 19mWorkshop - All Aboard The Elixir Express!