Data Integrity Tests in CI/CD Pipelines
summary
Commit triggers a pipeline that snapshots a production copy to staging, runs a DB migration, then fires a GitHub Actions integration test that validates row counts and business logic. An open‑source wrapper CMDhook runs the migration/webhook gating; Prometheus metrics via Prometheus drive a canary check and automatic rollback on excess warnings. Demo shows an induced test failure stopping the rollout, then a successful run promoting 6.0.2 to production. Notes cover anonymization via CSI snapshots and incremental replication for terabyte‑scale datasets.
AI-written summary · about
chapters (10)
- 0:00Data integrity in CI/CD
- 1:48Tightly coupled data services
- 3:12CI/CD pipeline workflow demo
- 6:09Database migration and staging
- 8:20Integration tests on production data
- 10:39Data anonymization in staging
- 12:50Production deployment with canary
- 14:18Web hooks in deployment
- 16:06Data integrity in Kubernetes
- 17:11Q&A
related talks
46mA Database DevOps Pipeline
01h 00mContinuous Integration and Delivery for Databases
39mRemove your Testing Bottleneck with CI/CD Pipelines
05mCI/CD Pipelines - Oh, the Places You'll Go!
26mIntelligent Deployment Pipelines
51mPatterns & Playbooks to Make Every Commit Production Ready
20mDataEngBytes 2023 PER-T1-10 - Matthew Carter
54mA Database DevOps Pipeline