Replacing Cassandra: A Digital Transformation for the World’s Largest Digital Identity Network
summary
ThreatMetrix replaces a 96-node Cassandra cluster with 28 Aerospike nodes, cutting median end-to-end latency from ~100–120 ms to ~30 ms while handling ~130 million transactions per day (3 billion/month) for 1.4 billion accounts. Data grows from ~110 TB to ~200 TB per DC, planning 48 nodes and ~1.3 PB flash. Daily drive writes fall to about a quarter, SSD endurance improves, and migration uses a custom SSTable reader with double-writes into both clusters. Highlights: 96→28 nodes, 100ms→30ms, 130M tx/day, SSTable reader and double-write migration.
AI-written summary · about
chapters (13)
- 0:00AEROSPIKE USER SUMMIT 2018
- 6:10Universal fraud & authentication decisioning in real time at global scale in the cloud
- 8:29Real-Time Decisioning Example: Identifying trusted returning customers
- 10:29General Requirements
- 11:18Storage Requirements
- 12:42Performance Requirements
- 16:32The new storage layer
- 16:56Migration
- 18:39The Path To Better Performance
- 20:22Unexpected benefits
- 24:47Cluster Growth (per Datacenter)
- 26:15Threat Metrix real-time API architecture
- 27:53Real-world performance
related talks
31mUser Attribution at Scale: All-Flash, ½ Trillion Objects
36maccelerating the digital agenda
33mSmartThings First 8 Years with Cassandra
26mHow Cassandra 4.X Help in Cutting Cost Mission
23mMyntra's Sub-Millisecond Journey with Aerospike
36mStories of Cassandra in Adobe Audience Manager
40mLatency, Throughput & Fault Tolerance: Building a Modern Database
35mThe What and Why of Cassandra Datamodelling at Scale