Go Channels slow down with more CPUs
summary
Fastly finds a surprising 3–10x drop in Go channel throughput as CPU count rises: benchmarks on real aggregator hardware (up to 128 cores) show per-channel rates falling to ~1M msgs/sec from ~6–10M. Tests use Go benchmarks, benchstat and varying CPU caps; range vs select behavior and buffered batches are measured. Workarounds include lowering GOMAXPROCS (e.g. GOMAXPROCS=32), batching messages, sharding across multiple channels, timeouts, or more workers/containers.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:26Speaker background Fastly edge
- 3:58Fastly caching shielding reduce load
- 5:47Logging bottlenecks
- 7:33Channel performance plateau
- 10:03Channel locks in Go runtime
- 11:30Single channel bottleneck logging
- 13:56Hardware specs benchmark setup
- 15:26Benchmarking tips Go tests
- 17:55Parallel benchmarks hardware
- 19:00Benchmark results slowdowns
- 21:26Contention in channel ops
- 22:48Buffered channels throughput
- 24:12Key findings throughput drops CPUs
- 24:34Solutions channel performance
- 25:14Solution adjust GOMAXPROCS
- 26:18Solution lock goroutines threads
- 27:20Solution timeouts channel sends
- 28:44Solution split load channels
- 29:28Solution buffer messages
- 31:13Containerized workloads
- 32:18Summary findings solutions
- 33:23Q&A CPUs worsen throughput
- 34:27Q&A Performance fluctuations
- 36:29Q&A Hyperthreading CPU pinning
- 38:19Closing remarks
