BERT and Ernie: Scaling your Ruby site with Erlang
summary
Moves GitHub off shared GFS onto 17 commodity Rackspace machines and replaces shared disk with a proxy that uses content‑aware routing via proxy_machine + Redis so frontends talk to the right file server. Introduces BERT (Binary Erlang Term) serialization and Ernie server (BERT‑RPC) to avoid IDLs/codegen, support streaming, casts and simple RPCs. Benchmarks hit ~2476 req/s in a 5,000‑call test and the system already serves ~626 million requests, eliminating shared‑FS outages and proving the approach works.
AI-written summary · about
chapters (15)
- 0:00Welcome
- 4:44NIH syndrome debunked
- 16:49Scaling GitHub solutions
- 17:11Engine Yard scaling
- 18:55Rackspace disk system
- 20:21Proxy Machine routing
- 22:28Simple RPC choice
- 27:49Erlang concurrency
- 32:27Introducing Bert
- 35:36Bert RPC
- 41:59Bert and Ernie implementation
- 43:25Bert RPC demo
- 46:10Benchmarking results
- 47:12Contributing to Bert and Ernie
- 47:56Closing remarks
