Running Low-Latency Workloads on Kubernetes
summary
Practical playbook for running SpiceDB-style, single-digit-millisecond network services on Kubernetes: shows benchmarks (hit ~1M QPS, P95 ≈5 ms), concrete code and infra tactics — profile hot paths, reuse allocations and use Go pools, tune GC with GOMEMLIMIT env var, serve from memory using tinyLFU cache, de‑duplicate requests with singleflight, use hedging carefully, pin CPUs with the static CPU manager, use taints/tolerations and Carpenter autoscaling, and wire memory limits via the downward API.
AI-written summary · about
chapters (20)
- 0:00Low-latency systems on Kubernetes
- 1:49Low-latency workload requirements
- 3:58SpiceDB and its use case
- 5:02Designing for understandability and profiling
- 6:48Concurrency pitfalls and CPU over-subscription
- 7:51Memory management in low-latency systems
- 8:33Parallelizing workloads for lower latency
- 9:34Hedging and single-flight techniques
- 11:00Serving data from memory and pooling
- 12:24Go memory limits and GC
- 13:07Reusing allocations with object pooling
- 14:09Optimizing GC with data structures
- 15:14Low-latency workloads on Kubernetes
- 16:15Kubernetes defaults and limitations
- 17:18Taints, tolerations, and affinity
- 18:42Resource requests limits trade-offs
- 20:28Context switching and scheduling
- 22:37Static CPU manager and autoscaling
- 24:45Isolation model for performance
- 26:50Q&A on Go memory limits and SpiceDB
related talks
15mOptimizing Application Performance on Kubernetes
02h 26mKubernetes Performance Tuning Workshop
47mOptimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latency
31mLoad Testing Kubernetes: How to Optimize Your Cluster Resource Allocation in Production [I]
36mOptimizing Kubernetes for Low-Latency and Real-Time Containers
25mTune It Up! Enabling Low Latency in Kubernetes Clusters
19mOpenShift for low latency and real time workloads
27mThe Kubernetes Hunger Games: Distro Performance in the Edge