Adventures in Rate Limiting: Spotify’s Journey Writing a Scalable Envo...
summary
Spotify builds a distributed Envoy rate limiter called Time Cop as a Go Envoy filter with Java aggregators, using consistent hashing to assign buckets and periodic batch reports so envoys send tens of aggregator requests per window instead of millions. Blocking actions flow via Pub/Sub to all aggregators; P50 latency rises ~3 ms and P99 ~11 ms in benchmarks. It works in production for perimeter IP/user limits, but the Go filter causes operational debt and is being ported to C++ for long‑term reliability.
AI-written summary · about
chapters (8)
- 0:00Introduction to rate limiting
- 2:24Evaluating existing solutions
- 7:26Designing Time Cop
- 10:42Time Cop architecture
- 14:02Choosing an implementation
- 19:18Go development experience
- 23:35Production and final thoughts
- 25:52Q&A and concluding remarks
related talks
24mHow Spotify Migrated Ingress HTTP Systems to Envoy
36mHow Spotify Migrated Ingress HTTP Systems to Envoy
10mLightning Talk: Rate Limiting With Quotas Is Hard - So We Didn't!
28mManaging and Scaling Real Time Data Pipelines using Go
31mHow We Moved Spotify To a Proxyless gRPC Service Mesh
26mIntegrating Stateful Load Balancing At Scale In Envoy
06mLivin' on the Edge (of your cluster)
05mGo for Real Time Streaming Architectures