Scaling Venmo for High Volume Events
summary
Venmo runs a production‑like performance test cluster, injects traffic with locust.io on Fargate, and observes results in Datadog to find bottlenecks before events. The stack uses Amazon Aurora (with read/write separation and RDS Proxy for writers), DynamoDB, DocumentDB, ElastiCache (sharded or non‑sharded), MSK/Kafka, Cassandra and OpenSearch; autoscaling and HPA handle normal peaks but pre‑scaling and sharding handle sudden spikes for a service with 66M monthly active users and sub‑second latency goals.
AI-written summary · about
chapters (9)
- 0:00Introduction and speakers
- 1:04Venmo's growth and evolution
- 7:05Architecture challenges and scaling
- 11:53Performance testing strategies
- 15:20Relational data store scaling
- 19:29Caching solutions and trade-offs
- 22:17Search and OpenSearch scaling
- 24:25High-volume event preparation
- 27:17Q&A on database scaling
related talks
47mScaling: from 0 to 25 million users
06mProject Lightning Talk: Vitess in Production: Powering Applications at Massive Sca...
25mScaling Data Platforms: Managing Growth from Zero to Billions of Transactions
38mHow to handle 100k events per second
47mScaling: from 0 to 25 million users
44mIntroduction to application scaling
15mPlatform Engineering for FinTech
51mLarge-scale fault-tolerant on prem deployments powering 100M+ transactions a day