Tradeoffs in Resiliency: Managing the Burden of Data Recoverability
summary
Five years of Google data-integrity work shows concrete tradeoffs: redundancy is not durability because bad writes replicate; transaction logs often only replay forwards so point-in-time snapshots are needed; backups frequently sit in cold, remote storage requiring bandwidth, staging disk and DB CPU to unpack. Building targeted tooling for partial restores and automated validate restores turns restores into recoveries and enables proactive consistency checks, which measurably shortens time-to-recovery in the team’s risk sheet comparisons for incidents like 1% corruption or truncating the top 5% largest entries.
AI-written summary · about
chapters (17)
- 0:00Start
- 2:50Advanced mode: Large scale data
- 3:30Redundancy is not durability
- 5:30Where are the backups?
- 6:23Are you sure the backup contains what you expected?
- 7:07Do you have room to unpack the
- 15:37What are you storing?
- 16:17Know your storage
- 16:48Consider your recovery options
- 18:07Different datastores, different
- 20:01Prepare Validations
- 22:25What guides SRE choices?
- 23:17SLOS 6 Error Budgets
- 25:23Explore Improvements
- 28:05Some Risk vs. Cost Tradeoffs
- 30:01Wrap up
- 30:47SRE resources
related talks
20mHow Can We Make Data Integrity Easy?
47mDisaster Recovery Strategies for SQL Server- 47mDisaster Recovery Strategies for SQL Server
46mLock, Stock and Backup: Data Guaranteed
29mYou Don't Need a Database Backup Policy
11mProtect and Recover with Autonomous Recovery
45mChasing unicorns : 0 dataloss and 99.999% availability
38mProtecting Database Backups From Ransomware and Malicious Intent