What's the cost of a millisecond
summary
Shows that a 1 ms delay deep in the stack can amplify into hundreds of milliseconds — example arithmetic 16 ms becomes 740 ms because of queuing and retries, with per-hop amplification often ~200% and worst cases up to 100x. Demonstrates with queueing theory and simple simulations, cites Node.js MySQL default 10 s timeout and AWS AZ RTT ≈ 0.25 ms. Recommends practical fixes: set dynamic timeouts, use a timeout budget, apply speculative execution and load shedding to control amplified latency.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:12Latency amplification explained
- 3:19Why latency matters
- 3:40Q Theory crash course
- 8:17High utilization risks
- 10:25Head-of-line blocking impact
- 12:32Sources of amplification
- 13:57Timeouts and retries
- 16:51Fork join pitfalls
- 18:17Microservices performance cost
- 19:00Setting proper timeouts
- 20:48Timeout budget strategy
- 21:54Parallel dispatch benefits
- 22:59Speculative execution
- 24:06Probabilistic retries
- 24:47Capacity and latency management
- 25:31Reducing variance importance
- 26:33Cost of a millisecond
related talks
30mWhat's the Cost of a Millisecond?
01h 01mWeb Performance Deep Dive
14mHarnessing Cascading Timeouts for System Resilience
16m"Latency: Why You Should Worry and What You Can Do About It"
21mIn the Blink of an Eye
49mEvery Millisecond Counts Performance & UX
49mFrom Milliseconds to Millions: The Numbers Driving Web Perf
34mOnce More with Feeling