Real Time Performance at Massive Scale
summary
Machine Zone describes an Erlang-based realtime stack that handles spikes from 5M to 40M DAU, sustaining ~100–500k concurrent users and broadcasting to 3.5M sockets under 200ms in benchmarked rooms. Built components include a custom message queue, sharded MySQL + Redis clusters, Lua consumers inside Erlang, and a real-time machine translation pipeline for 32 languages; the live system processes about 2 billion events per day and runs without full restarts, with automatic failover and linear scale-out.
AI-written summary · about
chapters (29)
- 0:00Intro
- 0:33Skeleton Scalability
- 1:48Consistency Availability Partition Tolerance
- 4:37Single Point of Failure
- 7:12Challenges
- 10:30Map
- 10:52Chat
- 11:47Scaling
- 12:45Language Barrier
- 13:13Machine Translation
- 14:35Game Demos
- 16:39Secret Sauce
- 18:27Profiling
- 19:45Why Erlang
- 21:21Message Queueing
- 21:43Group Chat
- 22:50Transfer Transformation
- 23:15RealTime Events
- 24:32Mass Push
- 25:04Message Queue
- 26:15Python Lua
- 27:02PHP
- 29:35SQL
- 29:56Master Master Slave
- 31:40Data Sharding
- 34:32Topology
- 35:00Benchmarks
- 36:50Alternatives
- 39:48Machine Zone
related talks
35mScaling Erlang cluster to 10,000 nodes
44mKeynote: Erlang Powered Rabbits
48m"Handling Millions of Concurrent Users with Erlang OTP"
01h 01mThe Erlang Ecosystem
37mReactive Architecture on AWS: from 0 to 1.6M events/minute
38mCrunching data in real time for Fortune500 clients
19mScaling to 500k concurrent customers with orleans
41mTracing Real-Time Distributed Systems