Capacity Planning with the Universal Scalability Law
summary
Introduces the Universal Scalability Law and shows how to fit its three parameters to real monitoring or load‑test data using nonlinear regression (scipy.curve_fit or R nls) to get actionable capacity numbers. Case study: Elasticsearch bulk indexing fits to sigma=0.0121, kappa=0.00009 and exhibits a peak at ~30 nodes with retrograde throughput beyond that unless sharding/configuration changes. Another case: a write‑ahead log on one machine shows ~7% contention and tops out near ~20 threads. Also outlines using dashboards/APM/logs, load tests, and vendor data to debunk linear scalability claims.
AI-written summary · about
chapters (18)
- 0:00Introduction and motivation
- 1:04Universal Scalability Law overview
- 2:30Linear scalability explained
- 3:54Contention and serialization effects
- 5:41Andahl’s Law and contention modeling
- 8:16Cross talk and handshake problem
- 10:03Universal Scalability Law formula
- 11:31Applications of the model
- 15:28Determining model parameters
- 17:33Case study Elasticsearch bulk indexing
- 20:15Parameter analysis and results
- 21:53System configuration tweaks
- 22:15Case study write-ahead log
- 23:39Debunking vendor scalability claims
- 24:47Cloud cost projection and scalability
- 25:51Universal applications beyond servers
- 26:53Recommended resources
- 27:15Q&A session
related talks
20mManaging Capacity
28mCapacity Prediction in External Services
50mFast-track Kubernetes and platform rebuilds with observability
21m"Capacity Prediction" instead of "Capacity Planning":
44mPlanning for Overload
30mData @Scale Seattle- Sundaram Narayanan
53mScale your business in real-time using ML in Microsoft Fabric Real-time Analytics
41mDrowning on Metrics or why Jack could have fit