SIGs Aren’t Silos: A Case Study Into Solving Inter-Domain Problems In Kubernetes Development
Talkat KubeCon + CloudNativeCon North America 2022 · on Wednesday, 2022-10-26· at 15:25· Ambassador Ballroom (Room 360)·
summary
A Kubernetes lifecycle refactor (v1.22) enforces terminal pod states and stops removing IPs for evicted pods, which causes evicted pods keep IPs and lets the endpoints controller include terminated pods in Service EndpointSlices. kube-proxy usually masks the problem by treating those endpoints as unready, but some consumers read EndpointSlices directly and see ghost IPs, producing 404s and outages. The bug stays unnoticed for six months; the fix is to preserve readiness semantics during shutdown and add cross-SIG tests and coordination to prevent regressions.
AI-written summary · about
chapters (7)
- 0:00Start
- 3:41SIG Network and Node bugs
- 6:40Pod Lifecycle: Pod Phases
- 8:10Pod Lifecycle: Pod Readiness
- 8:44Pod Shutdown - Before Refactor
- 11:22Service/Endpoints Removal
- 18:41The lessons learned
related talks
26mWhat it's like to lead SIG in Kubernetes and team
31mKubernetes SIG Contributor Experience: A Look Back, a Step Forward
27mIntro: Kubernetes (PM) SIG
29mIntro: SIG PM (Kubernetes)
23mScaling Kubernetes in Regulated Environments Through Platform APIs
38mSIG Release Deep Dive
31mBridging the Gap: Gradual Istio Implementation for Your Platform
26mIstio Migration: 50k+ Pods, Minimal Impact, Lessons Learned