Building a Scalable Real Time Bidding Exchange
summary
Built a real‑time bidding exchange in Erlang that connects to a C++ ad server over Thrift and ZeroMQ, stores campaign JSON in MySQL and caches in ETS, and maps cookies via Couchbase; bottlenecks include a single ZMQ process and single gen_server to Couchbase that forced process pools and passive sockets and use of LHTPC socket pools; system sustains ~8,000 HTTP DSP requests/sec and ≈50 billion DSP requests/day after fixes and targeted tracing.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:17About Me
- 2:38What is RTB?
- 5:24RTB Exchange Architecture
- 12:04How Much does it Scale
- 14:11ZMQ Interface
- 19:10Pooling Resources
- 23:01Tracing and Debugging
- 23:12Bid Collection Time
related talks
44mAdvertising at the Speed of Light: DMP Platform
47mMonolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges
23mBest of Both Worlds: Building a Mirror-trading Platform on Top of ...
27mScalable distributed systems from grounds up in Elixir
45mSelling Food With Elixir
44mAn Evolution of a Betting Engine at William Hill
52mThe Continued Evolution of High Frequency Trading Systems
38mAdvanced API Design for Scalable Distributed Systems