Managing FoundationDB at Scale
Talkat FoundationDB Summit 2019 · on Monday, 2019-11-18· at 14:30· Grand Ballroom 5, Marriott Marquis San Diego Marina·
summary
Apple runs hundreds of clusters with over 100,000 FoundationDB processes and manages them on Kubernetes using fdb_monitor, configMaps, anti-affinity and a custom sidecar-based live binary injection to update binaries and cluster files without full pod restarts. Operational loop covers bootstrap, recruit coordinators, exclusions, reconfigure, shrink/replacement and verification; upgrades use an all-at-once bounce strategy to limit recovery windows. An open-source Kubernetes operator implements this loop.
AI-written summary · about
chapters (15)
- 0:00Overview
- 0:50Overall Design
- 1:46Design Considerations
- 3:38Core Operation Loop
- 4:23Installing Binaries and Conf
- 5:42Add New Processes
- 8:09Change Database Configuration
- 9:43Removing Processes
- 11:17Change Coordinators
- 12:35Bouncing Instances
- 13:29Bounce Strategy
- 15:38Wait, what? Come on.
- 17:40Alternative: Rolling Bounces
- 19:00Alternative: DR Cutover
- 19:49Conclusion
related talks
26mRunning FDB at scale at Wavefront
24mCracks in the Foundation: Operating Cloud Foundry in the Real World
50mCracks in the Foundation: Operating Cloud Foundry...
13mKeynote: Chaos Engineering on Cloud Foundry
06mHow CFCR tackles Day 1& 2 Ops for Production Kubernetes
38mWhat Does It Take to Run Cloud Foundry at Enterprise Scale?
40mOperating a Self-Healing Bare-Metal Kubernetes Platform at Global Scale
29m#BuildingTheFuture: Keeping our Dutch Feet Dry: Setup and Operation of a...