What breaks when Aerospike hits 6 million QPS
summary
Runs Aerospike at multi‑million QPS for real‑time bidding, managing 13B user records on ~8 TB RAM across ~50 64‑core VMs with RF=2. At ~3M QPS 1–2% key busy errors cause P99 spikes and ~$50K/day lost; fix: switch writes to master-only eventual consistency (cuts workload ~60%, infra ~15%) and add circuit breaker + buffered writes. A DNS seed mismatch silently breaks writes during node migration. Large record sizes blow the 128 KB batch buffer, so move from batching to single lookups and hourly partitioning.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:01InMobi’s ad tech & Aerospike
- 4:11Hot key contention at 3M QPS
- 10:31DNS seed bomb failing writes
- 15:20Batching strategy costs
- 22:19Key learnings & reliability
- 24:47Q&A
related talks
22mThe Art of the Possible
49mJourney with Aerospike: Architecture Best Practices
50mAerospike Technical Overview
31mUser Attribution at Scale: All-Flash, ½ Trillion Objects
33mInnovation with Aerospike: How AppLovin Handles Billions of Requests per Day
48mRebuilding a RT Datastore: the design, deployment & performance story
40mDeveloping highly scalable applications with Aerospike
45mAerospike + Persistent Memory = Voila!