How to Tame A Mastodon: Lessons for PostgreSQL at Scale
summary
Describes running a 25TB OLTP PostgreSQL cluster with 15 replicas, 150GB/hr WAL and ~230M transactions/hr and the concrete fixes used: switch to PGBackRest async archiving to keep replicas from falling behind, scale out pgbouncer instances to remove the pooling bottleneck, convert serials to bigints during a logical migration, remove nested savepoints, tune autovacuum thresholds and maintenance_workers, prefer create index concurrently, use monthly partitioning for large time-series tables and reorder columns to cut row size.
AI-written summary · about
chapters (6)
- 0:00Introduction and scope
- 3:54Managing big table challenges
- 20:23High volume transaction issues
- 25:15Managing read replicas
- 31:23Analytics and summary
- 34:07Q&A session
related talks
30mManaging Postgres at scale: Challenges, Tools & Techniques
36mPostgres at any Scale
58mScaling Postgres
37mPostgres at Any Scale
51mScaling Postgres to the next level at OpenAI
26mBiggish Data With Rails and Postgresql
31mBiggish Data With Rails and Postgresql
46mHow we made PG Fitter, Happier, More Productive