Queueing Theory, In Practice: Performance Modelling in Cloud-Native Territory [I]
summary
Builds a simple single-core queueing model and derives W ≈ λ s^2 / (2(1−λ s)) then fits it to production microbenchmarks (single-core throughput ~2–3k req/s, single-digit ms latency) to predict the knee where latency explodes; model trained on low-util data matches withheld points. Uses simulation, histograms and tracing, and calls out variability and service-time reduction as highest leverage. For scale, applies the universal scalability law (α assignment, β coordination), shows 16-server experiments, and recommends two-random-choice and iterative partitioning (Sparrow, Facebook/Scuba style) to trade coordination cost for latency.
AI-written summary · about
chapters (4)
related talks
46mQueueing Theory in Practice: Performance Modeling for the Working Engineer
38mA Practical Look at Performance Theory
06mPractical performance theory
50mDesigning for Performance
30mWhat's the Cost of a Millisecond?
42mPredictive Load-Balancing: Unfair but Faster & more Robust
38mPractical Performance Analysis
30mLightning in the Cloud: A Study of Transient Bottlenecks on n-Tier Web Application