Fail-Safe BigQuery: Disaster Recovery Automation Techniques
summary
An open‑source framework called BQ Backup Manager automates table backups and policies at scale and is already generalized from large customer deployments, using Cloud Scheduler, Cloud Run dispatcher, Pub/Sub, snapshot or GCS export workers and a metadata aggregator. It extends BigQuery’s native time travel 7 days window, handles tens of thousands of tables, and works around limits such as the 100 snapshot concurrency limit while recording backup history and alerts into BigQuery.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:14Disaster categories
- 6:56BigQuery service failures
- 9:27Cross-region replication
- 11:56User-level backup automation
- 15:28Backup policies for tables
- 21:27Scalable backup architecture
- 27:32Disaster recovery recap
- 28:35Open-source resources
- 29:18Q&A
related talks
27mBackup and Restore: Discover Best Practices for a Distributed Database Backup Strategy
46mAzure SQL Database Business Continuity During Disaster
24mAutomate Data Management with Autonomous Database
47mDisaster Recovery Strategies for SQL Server
25mData Demons: Unleashing Chaos or Driving Performance?
29mYou Don't Need a Database Backup Policy- 47mDisaster Recovery Strategies for SQL Server
01h 01mAzure SQL Database-Business Continuity During Disaster