Lightning Talk: Introducing ORCA: Load Balancing With Endpoint Provided Load Data
summary
Envoy adds ORCA-based inline load reports carried in response headers or trailers (binary endpoint_load_report-bin, JSON or name/value formats) and EDS/LRS extensions for metric names; a new Client Side Weighted Round Robin policy parses per-response utilization (CPU, mem, rps, errors), recalculates weights on each report but applies updates periodically to avoid thrash, yielding improved endpoint utilization and higher QPS in heterogeneous fleets. Current alpha supports only 0–1 utilization metrics (not cost); gRPC has supported ORCA since 2022; stable release expected this year.
AI-written summary · about
related talks
18mOrca: Server-assisted Multicast for Datacenter Networks
10mLightning Talk: Envoy @ Netflix
17mZone Aware Routing With Per-Locality Load Awareness
29mgRPC load balancing and Service Mesh
05mLightning Talk: Optimizing Cluster Workloads: Cilium Envoy on DPU
08mLightning Talk: It’s 10pm, do you know what your proxy is doing? Understanding...
10mLightning Talk: Rate Limiting With Quotas Is Hard - So We Didn't!
16mKeynote: LLM-Aware Load Balancing in Kubernetes: A New Era of Efficiency