Performing Databases Migrations Without Overtime
Talkat JCON Europe 2023 · on Wednesday, 2023-06-21· at 10:00· Cinema 5, Im Mediapark 1, 50670 Köln, Germany·
summary
Demonstrates using Postgres.ai Database Lab Engine to validate migrations against real production-like data by creating thin clones of Postgres databases via ZFS copy-on-write so clones spin up in under 10 seconds for databases up to 2 TB. Shows live demo of spawning and resetting clones, running migrations, and aborting failures before production. Supports anonymization on the fly, exposes a REST API and CLI for CI integration, costs examples ($1k/mo DIY vs ~5x for RDS clone), Postgres only.
AI-written summary · about
chapters (21)
- 0:00Introduction and setup
- 1:45Speaker introduction
- 2:48Use case example
- 4:12Testing assumptions and edge cases
- 7:25Data schema evolution challenges
- 9:49Shift left methodology
- 11:35Problem with data migration assumptions
- 13:04Copying production data issues
- 15:36Cost of manual data copying
- 17:01Automation for migration testing
- 17:41Database Lab Engine overview
- 20:29Development environment integration
- 22:13Live demo introduction
- 23:00Production data snapshot
- 25:50Running migration in thin clone
- 27:16Resetting thin clones
- 28:41Wrapping up usage
- 29:44Automating data migration tests
- 31:06Q&A session
- 33:13Learnings and stakeholder buy-in
- 35:37Technical explanation
related talks
17mA Laboratory for Databases
20mOpen Source Database Operations in the Age of AI
06mLightning-Fast Database Tests
51mPostgreSQL on ZFS
46mTransform your development workflow with thin cloning
44mEfficiently Deploying PostgreSQL Instances: The Evolution
46mBig databases, many servers, data center or cloud - it's all here!
44mContinuous Database Administration