System Performance and Queuing Theory - Concepts and Application
summary
Queueing theory explains why systems with 80% utilization still see long delays: variability in arrivals or service makes queues form and average residence time grow nonlinearly, so halving idle time roughly doubles latency. Simulations and Prometheus/MongoDB data show 95th-percentile latencies exploding (at 90% utilization ~95th percentile ≈ 30× service time). Centralized queues feeding many workers outperform per-worker queues; tradeoffs include failure blast radius and granularity. Practical tools and models mentioned include a PDQ model and the Universal Scalability Law for capacity planning.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:50Why Queuing Theory
- 2:36Queuing Theory Basics
- 4:30Parts of a System
- 8:27Capacity and Utilization
- 13:29Queuing Equations
- 15:49Utilization and Latency
- 20:39Queuing Multiple Servers
- 25:11Practical Applications
- 25:41Disclaimer
- 28:00CPU Usage
- 29:24CPU Usage Graph
- 30:08Latency Per Query
- 32:41Expected Resonance Times
- 34:21Prometheus Queries
- 35:15PDQ
- 36:32Universal Scaling Law
- 38:01Recommendations
