Why Your Database Fails in AWS And It’s Not the Database’s Fault
summary
Two real incidents show the database was a victim, not the culprit: a nightly ETL exhausts GP2 burst IOPS exhausted, spiking IO wait and query latency until API timeouts; response is migrating to GP3 with provisioned IOPS, adding EBS+DB correlation, synthetic ETL load tests and capacity planning. A second outage comes from a bad WAF rule that triggers a retry loop, creating millions of writes, connection pool exhaustion and 99% CPU; fixes are WAF canary deployments and app-side circuit breakers plus correlated app/WAF/DB telemetry.
AI-written summary · about
chapters (5)
related talks
22mLessons from Automatic Incident Resolution for a Million Databases
26mHow to survive a Lambda-induced data tsunami in AWS
26mNavigating Managed Databases for Over a Decade as a Non-DBA
43mBest practices for continuous testing and observability
22mWhat happens when DynamoDB explodes? A practical guide
59mWhy Databases Cry At Night
40mSh... Fail Happens: Fail-aware Events Processing on AWS
34m5 Things I Learnt from Causing a Cloud Provider Outage