Operating A Multi Petabyte System Of Record For Billions Of Users
summary
Verizon Media modernizes Sherpa by migrating a multi‑petabyte, multi‑tenant system of record (≈200B records, ~1 PB) into Aerospike across 6 data centers, serving ≈5M reads/sec and ~300–400k writes/sec on ~400–500 beefy PMEM+NVMe nodes. Replication uses Aerospike XDR with bin-level convergence for cross‑DC concurrent writes, Pulsar connectors for ordered change notifications, strong per‑DC consistency, eventual cross‑DC convergence, a UDP rate limiter, mutual TLS, Yamas monitoring and Screwdriver CI; migration is halfway done and set‑indexing is planned to fix expensive scans.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:14What is Sherpa
- 3:07History of Sherpa
- 7:37Aerospike Architecture
- 9:31Multitenant Capabilities
- 11:06Data Replication
- 13:05Change Notifications
- 14:12Monitoring
- 15:19Security
- 16:35Operational
- 17:35Migration
- 18:50Set Index Stuff
- 25:06Persistent Memory
- 26:44Pulsar Connector
- 27:37Order Tables
- 28:41Cluster Consistency
related talks
29mWhat breaks when Aerospike hits 6 million QPS
40mDeveloping highly scalable applications with Aerospike
36maccelerating the digital agenda
49mJourney with Aerospike: Architecture Best Practices
33mInnovation with Aerospike: How AppLovin Handles Billions of Requests per Day
51mLarge-scale fault-tolerant on prem deployments powering 100M+ transactions a day
50mAerospike @ One of India's Largest Fintech Companies
22mThe Art of the Possible