Chaos testing of a Postgres cluster on Kubernetes
summary
Shows that Kubernetes operators can approach cloud DB automation but do not guarantee zero downtime, with measured outages of ~30s for a replica failure, ~27s when the primary is fenced, a 10s rolling update plus a 22s self-upgrade pause in the demo. Uses Zalando Postgres Operator, Chaos Mesh for fault injection, Patroni for leader election and health, and PgBouncer for pooling; includes a custom observability agent integrating with Prometheus at ~1s resolution that exposes why failovers and replica lag cause application errors.
AI-written summary · about
chapters (9)
- 0:00Postgres on Kubernetes intro
- 3:48100% availability impossible
- 4:31Replica under load isolation
- 7:25Failing primary instance
- 8:35Network partition primary-replica
- 10:20Connection pooler isolation
- 11:46Postgres cluster upgrade
- 14:23Key takeaways and tools
- 15:53Q&A
related talks
50mDemystifying Kubernetes for Postgres DBAs; A Guide to Operators
27mFrom Zero to Hero: Scaling Postgres in Kubernetes Using the Power of...
31mLightning Talk: Kubernetes as Your DBA
48mPostgreSQL in Kubernetes: Operation - Day 2
26mKubernetes from the Database Out
34mWill your PostgreSQL operator crack under chaos?
41mCloud-Native Databases in Kubernetes with OpenShift and Postgres
34mRun PostgreSQL:The Kubernetes way