Innovation with Aerospike: How AppLovin Handles Billions of Requests per Day
summary
Platform runs Aerospike as core storage and adds a Spark layer to make device data queryable at scale, supporting 70 billion ad requests per day across ~35 deployments and ~350 servers with ~250 billion objects and 700 TB of storage. Exports and backup/restore are rewritten to Parquet and fan‑out writers to hit a full cluster export in about 2.5 hours and run ~80% faster than asbackup. Moving small datasets into local Aerospike instances plus the v13 batch protocol yields ~213 million reads/sec and reduces memory by ~75%, delivering joins about 7x faster than Spark shuffle for disproportionate dataset sizes.
AI-written summary · about
chapters (17)
- 0:00Intro
- 3:52Our Aerospike Use Cases
- 7:41Speeding Up Development and BI
- 10:26Aerospike Backup/Restore with Spark
- 11:02BI & Debugging with ApacheZeppelin
- 13:23Aerospike Export Benchmarks
- 14:18Combing Spark and Aerospike
- 15:34Down Funnel Event Matching At Scale
- 17:37Spark to the Rescue
- 19:09Avoiding the Shuffle
- 20:41Proof of Concept
- 22:03Full Scale Implementation
- 24:36Peak Reads/s
- 24:55The Final Comparison
- 25:23Bringing it to the Masses
- 27:16Limitations and Future Work
- 29:09Aerospike and Spark
related talks
31mAerospike's Innovation Agenda
49mJourney with Aerospike: Architecture Best Practices
05mHighlights From Aerospike APJ Summit 2023
36maccelerating the digital agenda
50mAerospike @ One of India's Largest Fintech Companies
22mThe Art of the Possible
40mDeveloping highly scalable applications with Aerospike
15mAWS On Air Voice of the Customer ft. Aerospike, Inc.