Data corrupting architectures we know and love
summary
Common web scaling patterns let concurrent read-add-write race access a shared database counter and produce lost updates so a three-request scenario ends with two or even one instead of three because of last-write-wins behavior. Fixes shown include in-database transactions or stored procedures, distributed transactions or consensus (Paxos/Raft), locks or actor-style synchronization, event sourcing/CQRS and conflict-free replicated data types like CRDTs. Prefer sending increment operations rather than writing absolute values.
AI-written summary · about
chapters (21)
- 0:00Intro
- 2:11GENERALIZATIONS AHEAD
- 2:45OUR MOST POPULAR ARCHITECTURES CAN BE TRACED TO WEB APPPLICATIONS.
- 4:19OUR SIMPLE WEB APP IS A DISTRIBUTED SYSTEM
- 4:37DISTRIBUTED SYSTEMS AREN'T HARD. THEY'RE EASY
- 5:27SOME JUSTIFICATION AHEAD
- 7:46SPOT THE DATA CORRUPTION?
- 10:45WE'RE EXPERIENCING A DATA RACE
- 11:20OUR DATABASE IS A SHARED VALUE
- 11:48WHOLE LOTTA WRITING GOING ON
- 16:53LAST WRITE WINS IS A FANCY NAME FOR "LET'S CORRUPT SOME DATA".
- 17:47OUR MOST POPULAR ARCHITECTURES FOR SCALING
- 18:21FUNCTIONS AS SERVICE
- 19:58ALL FEATURE DATABASE AS A SHARED VALUE AND CONCURRENT WRITES
- 20:13ALL HAVE DATA CORRUPTION BAKED INTO THE CORE ARCHITECTURE
- 23:06STORED PROCEDURES AND TRANSACTIONS
- 23:35DISTRIBUTED TRANSACTIONS
- 23:51SYNCHRONIZE DATA ACCESS
- 25:02CONSENSUS ALGORITHMS
- 25:23CONCURRENCY FRIENDLY DATA STRUCTURES
- 25:51COMMUNICATE OPERATIONS NOT VALUES
related talks
42mA Race To The Bottom: Database Transactions Undermining Your AppSec
28mPractical Approaches To Scaling Distributed Transactions
45mTransactions and Concurrency Control Patterns
41mConcurrency, Scalability and Transactions -- Myths and Surprises
29mModern Database Architecture: From SQL to NoSQL & Distributed SQL
52mTransactions: Myths, Surprises and Opportunities
47mBeing transactional in 2019 - what does this actually mean?
50mCRDTs in Practice