What's the Cost of a Millisecond?
summary
Claim: tiny delays amplify everywhere because queues, head‑of‑line blocking and retries compound multiplicatively. Uses an M/M/1 simulation, Kingman’s formula and a demo tool called cumulator to show higher variance or utilization explodes wait time (e.g. 50%→55% roughly doubles amplification; 90%→99% can increase it tenfold). Recommends concrete fixes: set timeouts from histograms and use a timeout budget, cap queue length and concurrency, apply backpressure, and prefer speculative or parallel dispatch over naive fork‑join. Reports single‑millisecond backend blips turning into 10–50 ms at p99.
AI-written summary · about
chapters (13)
- 0:00Introduction to CLDB
- 0:52Defining Latency Amplification
- 2:20Queuing Theory Basics
- 5:06Simulation and Behaviors
- 7:35The Latency Curve
- 9:48Cross-Task Impact
- 12:47Real-World Latency Sources
- 14:20Timeouts and Retries
- 17:53Combating Amplification
- 20:44Advanced Performance Patterns
- 23:04Capacity and Backpressure
- 25:49Reducing Variance
- 28:25Summary and Conclusion
related talks
27mWhat's the cost of a millisecond
46mQueueing Theory in Practice: Performance Modeling for the Working Engineer
30mQueueing Theory, In Practice: Performance Modelling in Cloud-Native Territory [I]
38mA Practical Look at Performance Theory
45mUnderstanding Microservice Performance
21mIn the Blink of an Eye
34mOnce More with Feeling
52mMeasuring and Optimizing Tail Latency