The Computer Wants to Lose Your Data
summary
Databases must cope with torn 16KB (MySQL) or 8KB (Postgres) pages versus 4KB atomic sectors, so MySQL uses a doublewrite buffer that guarantees one intact copy on disk; Postgres changes to crash-on-error after the 2018 fsync returns EIO once discovery and kernel fixes across 4.13–4.17; many NVMe/SSD devices have volatile write-back cache lies that claim flush succeeded but lose data, and turning caches off fixes safety at a large performance cost.
AI-written summary · about
chapters (8)
- 0:00Introduction to data storage
- 2:20MySQL doublewrite buffer
- 9:35Atomic writes in hardware
- 12:06Postgres fsyncgate incident
- 21:15Database error handling
- 22:12Hardware disk caches
- 27:25Safe hardware strategies
- 30:42Reliability and conclusions
related talks
58mWhy Databases Cry At Night
53mSo, you decided to reliably write data to disk
48mA Deep Dive Into Database Internals
41mLinux IO internals for PostgreSQL administrators in 2023
28mHow Can Databases Capitalize on Computational Storage?
35mQuestioning Database Claims: Design Patterns of Storage Engine
59mWhy Databases Cry At Night
49mBuild a database from the ground up