That's 'Billion' with a 'B': Scaling to the Next Level
summary
WhatsApp runs Erlang R16B01 with about 400–500M monthly users, ~19B messages in and ~40B out per day, peaks near 150M concurrent and 230k connects/sec; infrastructure is ~550 servers and ~11,000 cores. Scaling choices include write-back cache with 20s sync to avoid IO spikes, gen_factory → gen_industry to remove dispatch bottlenecks, multiple Amnesia TMs and split Amnesia islands into two-node peers, round-robin async file ports and multiple timer wheels. Patches produce a 4x ETS hotspot speedup and generally hold under load, though a mass reconnect caused a major outage.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:41Erlang
- 2:32Numbers
- 3:10Multimedia Holiday Cheer
- 3:43System Overview
- 4:08Output scale
- 4:52Throughput scale
- 6:54Hardware Platform
- 9:00Software Platform
- 9:34Decouple
- 12:32Parallelize
- 19:54Optimize
- 25:13Patch
- 29:34Meta-clustering
- 29:48Topology
- 29:58Routing
- 30:12Clearing the minefield
- 32:34Questions?
- 39:09Challenges
related talks
35mScaling Erlang cluster to 10,000 nodes
43mHow WhatsApp moved 1.5B users across data centers
16mRelay Infrastructure at WhatsApp Scale
53mIdioms for Building Distributed Fault-tolerant Applications with Elixir
39m400 Billion Messages a Day with Kafka
39mConcurrency Oriented Programming In A Modern World
34mGrowth from zero to 15,000 messages per second: painful and instructive
48m"Handling Millions of Concurrent Users with Erlang OTP"