Our Journey Connecting Millions of Containers with gRPC... - Antoine Tollenaere & Sergey Matyukevich
summary
Datadog describes concrete fixes for large gRPC fleets: keepalive enabled by default cuts silent connection-drop detection from ~15 minutes to seconds, outlier detection ejects misbehaving replicas to eliminate error blasts, and switching pick_first to round_robin then to weighted round robin (using load reports) evens CPU across heterogeneous cores. Go servers share buffers via sync.Pool yielding a 40% memory reduction, Python uses a sidecar connection pooler, and random subsetting reduces connection counts while deterministic subsetting is under active exploration.
AI-written summary · about
chapters (10)
- 0:00gRPC at Datadog
- 1:25Why adopt gRPC early
- 3:37Keepalive for connection drops
- 5:21Outlier detection for misbehaving servers
- 7:12Load balancing with round robin
- 12:06Memory overhead from connections
- 16:01Connection pooling in Python
- 17:45Subsetting to reduce connections
- 19:55Future of deterministic subsetting
- 20:15Q&A
related talks
17mBuilding a gRPC Proxyless World: How Reddit Scaled Resilience...
28mDeep Dive: gRPC
30mgRPC & Kubernetes
41mKubernetes the Very Hard Way
44mKubernetes at Datadog Scale
35mKubernetes, Lessons Learnt
32mCruise’s Self-Driving Networking Journey
32mEnhancing the Performance Testing Process for gRPC Model Inferencing at Scale