Bigtable: A Journey from Binary to Service and the Lessons Learned along the Way
summary
Bigtable evolves from per-cluster binaries into a global multi-tenant service that runs for over a decade and powers 60+ internal projects, with tables that scale to trillions of rows and petabytes. Engineers build an automated resize path (config database → self‑service UI/CLI/API), quota enforcement, instrumentation, chargeback and capacity forecasting; early quota rules cause outages until validation and rollback protections are added. SRE and dev roles converge: shared on‑call, joint testing for multi‑tenancy, and SLOs like 1KB reads at 6ms when provisioned guide tradeoffs.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:54What is Bigtable?
- 2:00A user's view of Bigtable
- 4:22Initial Deployment Model
- 5:05Chaos Ensues!
- 7:52How hard can this really be?
- 8:10But how many servers do we run for each cluster?
- 10:45Evolution of a Service (Simplified version)
- 13:24Another customer wants to turn up on a new continent?
- 16:28Now we've got two systems!
- 18:17Customer Story: Turnup a New Cluster
- 21:40Engage with your customers
- 24:24Don't be afraid to change the services itself
- 24:56Even Small Changes are important
- 26:41Build a relationship with your counterpart team
- 28:16Share Priorities / Plan Jointly
- 30:11Share Operational Responsibilities
- 31:19Work together on new features
related talks
09mExplains the Cloud Bigtable Platform
05mNext Big Step
08mScaling and innovating with databases
25mPlatform Engineering for Modern Data Infrastructure
31mGoogle Cloud Bigtable: petabyte-scale NoSQL database for scalable computation
52mBuilding the World's Largest Enterprise Data Warehouse with BigQuery
26mFrom datasets to tables in a multitenant data lake
19mBuild Global-Scale Apps Using Globally Distributed Autonomous Databases