Large-scale fault-tolerant on prem deployments powering 100M+ transactions a day
summary
PhonePe runs on-prem with 50+ Aerospike clusters per site across three sites supporting ~380M registered users and a 100M transactions/day run rate; edge clusters serve ~400k QPS for sub‑millisecond reads while the data platform hits ~650k QPS using Aerospike plus HBase, MariaDB and Elasticsearch. Deployment uses bi‑directional XDR with user pinning for active‑active locality, custom protobuf integer encoding plus Aerospike compression saves ~47% storage, and operations face large records, high‑watermark disk pressure, node failures (~1 per 15 days), config/XDR errors and painful rolling restarts.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:11PhonePe scale and on-prem
- 4:00Technology stack and expansion
- 7:08Edge workloads and Aerospike
- 8:11Data platform and Aerospike
- 13:06Promise engine and Redis
- 17:14Data platform architecture
- 22:13Fraud detection with Aerospike
- 29:16Data platform challenges
- 33:54Multi-site deployments
- 40:11Multi-site Aerospike challenges
- 44:08N-star topology and migrations
- 48:03Database proxies and modeling
- 49:06Monitoring and config
- 50:11Conclusion and Q&A
related talks
50mAerospike @ One of India's Largest Fintech Companies
18mHow Microservices and a Key-Value Store Transformed India into a Cashless Society
33mInnovation with Aerospike: How AppLovin Handles Billions of Requests per Day
29mWhat breaks when Aerospike hits 6 million QPS
06mSnapdeal and Aerospike
23mMyntra's Sub-Millisecond Journey with Aerospike
05mHighlights From Aerospike APJ Summit 2023
40mDeveloping highly scalable applications with Aerospike