Zero-Copy Chaos. When everything you knew about tokio...
summary
System goes from 10M to 45M req/s and drops 30% while P99 spikes to 450ms → 8ms after a 72‑hour sprint that eliminates every allocation in the hot path. Concrete fixes include pooling with Bytes-backed ring buffers, IOvec scatter/gather writes, splitting runtimes for IO vs compute, replacing semaphores with work‑stealing backpressure, targeted unsafe blocks with benchmarks, and tools that mattered: Tokio Console, heaptrack, perf and jemalloc tuning, resulting in zero allocations on hot path.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:57Production collapse under load
- 2:04Traffic spike scale
- 3:11What failed and why
- 4:55Tools for hidden bottlenecks
- 6:22Zero copy as discipline
- 7:47Code before and after
- 9:13Unsafe with benchmarks
- 10:57Tokio defaults mismatch
- 11:38Runtime splitting backpressure
- 12:24Timer wheel optimizations
- 12:46Counterintuitive performance insights
- 13:33Task allocation pressure
- 14:17Unsolved challenges
- 14:40Final results
- 15:45Summary
related talks
05m“Using async-profiler to Find Tail Latency Issues”
06mZero-copy Transfers with io_uring and Rust
50mHigh Performance Network Programming in Rust
33mReaching Ludicrous Speed
29mI/O in Rust: the whole story
39mMemory Efficient Java
51mThe Quest for Speed: Journey to 50% Better P99 Times with Go
45mThe Joy of Building Large Scale Systems