Follow the Advice, but Not Too Closely - Building the Cluster YOU Need
summary
A 20 PB object cluster built from 26 storage units (each ~60 × 22 TB drives, >1 PB raw) uses 24 active nodes with two hot spares reserved to avoid multi‑day rebuild capacity loss, runs on a 400 Gbit backbone with 100 G links using Broadcom NICs, shifts from KVM guests to Ceph ADM containers and exposes data via RADOS Gateway with a four‑node index pool, drops CPU specs to 64 cores per storage unit after seeing low OSD CPU use, and proves performance in benchmarks.
AI-written summary · about
chapters (9)
- 0:00Introduction and background
- 1:45Community advice and documentation
- 3:09CPU optimization in storage clusters
- 4:14Cluster capacity and rebuild strategies
- 5:17Designing cluster requirements
- 6:45Operating SEF: self-support options
- 8:08Automation and knowledge sharing
- 9:12Failure conditions and testing
- 9:54Multiple clusters and scaling
related talks
11mKeynote: From Busy to Impactful: Maximizing Ceph’s Collective Resources
41mHopelessness and Confidence in Distributed Systems Design
53mPanel Discussion - Performance & Scalability
29mDowntime Budgets
34mReliability A to Z
52mPersistent SRE Antipatterns: Pitfalls On the...
27mLessons Learned From Architecting the Highest-scale Operational Systems in the World
30mManaging Kubernetes: What You Need to Know About Day 2 - Craig Tracey, Heptio