Service Discovery Challenges at Scale
summary
Dropbox describes a service discovery system named Sasanian that uses Zookeeper plus a per-host local daemon to multiplex clients and cache reads, coalesces state into single snapshot writes to avoid thundering herd and session storms, and runs a separate aggregator control plane that converts host snapshots into a service index. It leverages Zookeeper read-only mode, local caches, black‑box and white‑box monitoring, and chaos tests (killed ZK, aggregators) to measure propagation latency and availability.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:18Susanin Dropbox Service Discovery
- 2:09Zookeeper Backend
- 3:22Scalability
- 5:06Common approach
- 8:19Summary
- 10:15Scaling number of instances
- 15:32Integration with gRPC
- 17:43Key metrics
- 20:04Blackbox monitoring
- 22:04Whitebox monitoring
- 23:25Disaster Recovery Testing
- 24:29Conclusion
related talks
30mDistributed system design
15mBuilding a global and resilient streaming service API
48mService Discovery using CRDTs
33mDistributed Scheduler Hell
16mHijacking Service Discovery to Simulate Dependency Degradation
01h 08mDystopia as a Service
27mEmpowering Resilient Scalability: The Power of Choreography in Distributed Systems
25mEventually Consistent Service Discovery