Building Financial Systems on Eventually Consistent DBs

on YouTube ↗26m

" summary

Billing platform for Netflix migrates from MySQL to Cassandra to gain horizontal scale and multi‑region availability for 125M+ customers, trading ACID for latency and complexity. Engineers use lightweight transactions and local quorum to limit conflicts, generate keys with UUID v5 invariants to dedupe SQS at‑least‑once delivery, and replicate writes to Kafka plus nightly Hive snapshots for OLAP. Result: 99% of traffic on the new system by July 2018, roughly 2× the processed volume in half the time, with operational monitoring to catch inconsistencies.

AI-written summary · about

" chapters (23)

1. 0:00Intro
2. 1:28Billing Engineering
3. 3:53Netflix operates out of multiple AWS regions to ensure availability
4. 4:46Single Master Single Point of Failure
5. 5:05Read replicas
6. 5:33Slave Promotion
7. 6:07We needed something better...
8. 7:16But it's not perfect, especially for a financial system
9. 7:53So how do you build a financial system on top of it?
10. 8:31Lightweight Transactions in Cassandra
11. 9:22Update Consistency With Optimistic Locking
12. 10:08Two Scenarios
13. 10:45With Cassandra, last write wins
14. 11:40How does Cassandra achieve fault tolerance?
15. 12:21Local Quorum vs Each Quorum
16. 13:29Insert Consistency
17. 15:12Some things to consider
18. 16:20Minimizing inconsistencies
19. 18:38Building a financial system on top of Cassandra
20. 19:44We built a poor man's relational schema
21. 20:16Understand performance implications before using collection types
22. 22:13Querying across rows
23. 23:06Recap

" related talks

• 46mCassandra Serving Netflix @ ScaleVinay Chella · ApacheCon North America 2017
• 49mData Platform Architecture, Evolution, and PhilosophyJustin Becker, Kurt Brown · @Scale 2014
• 35mThe What and Why of Cassandra Datamodelling at ScaleChandi Datta, Sunil Dabre · Cassandra Summit 2023
• 32mMigrating Workloads from Mainframes to Cassandra - Problems, Experience and LearningsVenkat Mopuri, Seshu K Guddanti · Cassandra Summit 2023
• 34mHow Netflix Delivers Key-Value and Time-Series Storage at Any ScaleJoseph Lynch, Vidhya Arvind · Cassandra Summit 2023
• 46mApache Cassandra: 8 years in productionVladimir Degterev · JPoint 2024
• 45mMigrating Mission Critical Workloads from Mainframe to Open...Venkat Mopuri, Seshu K Guddanti · Critical Software Summit @ Open Source Summit North America 2023
• 35mA Customer’s Journey: Migrating from NoSQL to a Cloud Native Distributed SQL DatabaseSRIRAM SITARAMAN · Distributed SQL Summit Asia 2022