Beyond 1001 Dedicated Data Service Instances
summary
Claims a practical path to run thousands of dedicated DB instances by delegating VM orchestration to BOSH director and using a generic service broker with a service‑plugin (SPI) to produce per‑instance manifests. Experiments create up to 25 concurrent creates, measuring BOSH task lead/cycle time and system metrics; small vSphere shows up to ~17 minutes per instance under load, while AWS is more stable (~5 minutes). Target is 3 minute per create (≤9 min under peak); main bottlenecks are BOSH workers and underlying infra, not the broker logic.
AI-written summary · about
chapters (12)
- 0:00Introduction to scaling data services
- 1:24Design patterns for data services
- 5:15Shared cluster approach explained
- 8:52Challenges with shared clusters
- 10:40Dedicated containers for isolation
- 13:06On-demand dedicated service instances
- 18:01Lean approach to scaling experiments
- 21:57Measuring bottlenecks in architecture
- 25:11First scaling experiment results
- 27:20Infrastructure impact on scaling
- 28:24AWS vs vSphere performance
- 30:10Next steps and conclusions
related talks
24mOn-Demand Service Provisioning with BOSH 2.0
30mStreamlining the BOSH Experience
28mManaging Multiple Clouds with a Single BOSH Deployment
30mOn-Demand Data Services with Cloud Foundry - Maria Ntalla & Simon Jones, Pivotal
28mManaging Stateful Docker Containers with BOSH
25mBOSH: The Simple Solution for Complex Systems
25mBOSH Bootstrapping
28mHow to Deploy CF and Other Bosh Releases on Bare-Metal Environment