Intelligent Load Balancing in Kubernetes
summary
Databricks replaces kube-proxy/DNS data-path balancing with a lightweight, push-based control plane that distributes EndpointSlice updates via xDS push-based control plane to client libraries (Envoy and Armeria), enabling per-request Layer 7 routing. They implement Power of Two Choices plus zone-affinity and pod-health scoring, avoid a full service mesh for cost reasons, and report stabilized tail latency, resolved 4–5x pod QPS skews, and about a 20% smaller fleet after rollout.
AI-written summary · about
chapters (15)
- 0:00Databricks traffic challenges
- 3:10Request imbalance issues
- 5:55Kubernetes load balancing fails
- 8:47Initial solutions limits
- 13:47Intelligent load balancer requirements
- 16:13Custom load balancing design
- 20:04Endpoint discovery service
- 21:51Round robin client-side
- 23:15Cold start problem
- 25:20Beyond uniform distribution
- 26:25Bias from overloaded pods
- 28:56Power of two choices
- 31:01Results and benefits
- 32:43Key takeaways
- 34:07Q&A
related talks
26mIntegrating Stateful Load Balancing At Scale In Envoy
37mBuilding the Multi-Cluster L7 Load Balancer
16mKeynote: LLM-Aware Load Balancing in Kubernetes: A New Era of Efficiency
38mThe Untapped Power of Services - L7 Load Balancing Without a Service Mesh
13mEdgeBalance: Model-Based Load Balancing for Network Edge Data Planes
46mKeeping the Balance: Load Balancing Demystified
23mBuilding Unified Global Load Balancing for the Edge With k8gb
11mLightning Talk: Scaling Kubernetes Global Balancer with Crossplane - Yury Tsarev, Absa Group