Lightning Talk: More Intelligent Global Rate Limiting
summary
Google builds a global rate‑limit service for Envoy that replaces per‑request unary gRPC with a bidirectional gRPC stream, groups clients into quota‑bucket IDs, and caches assignments in a thread‑local cache. The service targets millions RPS, removes one‑RPC‑per‑request bottlenecks, reduces backend synchronization, and supports real‑time quota reallocation (e.g., shifting capacity to a shopping site during Black Friday). Design is implemented across teams and slated for rollout early next year.
AI-written summary · about
chapters (4)
related talks
32mDoorman: Global Distributed Client Side Rate Limiting
26mTraffic Management w. Rate Limiting to Ensure API Scalability & Resiliency
10mLightning Talk: Envoy @ Netflix
42mStop Rate Limiting! Capacity Management Done Right
35mEvolving Critical Systems With gRPC: Bidirectional at Scale — What Works, What Hurts, What’s Next?
10mHow Tinder implemented Envoy global rate limiting at scaleVirtual
26mAdopting gRPC for Low Latency, High Bandwidth Scenarios
47mMonolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges