Deploying Django migrations at Kraken scale with Tim Bell
summary
Kraken runs ~350 Django apps with over 9,000 migration files, 23 production instances across eight time zones, some tables with billions of rows, and ~150–200 deployments a day, so migrations must avoid long locks and rewrites. The concrete techniques are splitting changes into add constraint not valid then validate constraint, using create index concurrently with Atomic=False or the safer migrate_with_timeouts command from Django PG migration tools, adding backfill jobs that batch and pause, and automating or faking migration state where needed; these tools and processes mostly work but remain a work in progress.
AI-written summary · about
chapters (12)
- 0:00Django migrations at Kraken scale
- 1:27Common deployment issues
- 3:16Migrations without downtime
- 7:10Slow migrations into steps
- 10:00Adding indexes safely
- 12:09Backward-compatible migrations
- 14:14PostgreSQL locking hazards
- 16:21Create index concurrently safely
- 17:46Kraken scale challenges
- 19:32Failed migrations at scale
- 20:54Backfilling large tables
- 23:24Conclusion
related talks
33mPeaceful Django Migrations with Efe Öge
29mMigrating Data Quickly and Safely
21mMultitenant, the Hell of Migrations
26mDjango migrations, friend or foe? Optimize them for testing
33mAdding zero-downtime migrations strategy in SaaS project
40mrails db:migrate:safely
30mForklifting Django: Migrating A Complex Django App To Kubernetes
21mMigrating to Success: Unleashing the Power of Django Migrations