Auto Operation of Hundreds of K8s Clusters in Yahoo! Japan
summary
Kubernetes-as-a-Service on-prem runs on OpenStack using CRDs and custom controllers that poll the OpenStack API to create, drain and replace VMs, achieving auto rolling upgrades with zero downtime for control plane and workers. Design favors one service per cluster for security; the platform enforces MachineDeployment-style resources, auto-healing and fast patching (vulnerabilities rolled out within 24 hours). Scale numbers: about 400 clusters and 4,000 machines managed by a roughly ten-person team.
AI-written summary · about
chapters (15)
- 0:00Start
- 0:51Agenda
- 3:03About Yahoo Japan Corporation
- 4:24Cluster Management Team
- 5:16Our mission
- 6:07Why Kubernetes as a Service
- 6:45Requirement
- 7:58About Kubernetes as a Service
- 10:19Cluster configuration
- 11:51Features
- 14:11Automatic recovery from VM down
- 15:51Easy to update
- 20:18400 clusters/10 management members
- 21:42Planned functions
- 23:23Summary
related talks
35mAuto Operation of Hundreds of K8s Clusters in Yahoo!
33mRunning Istio and Kubernetes On-prem at Yahoo Scale
40mLarge-Scale K8s Cluster Operation and Management
03mFrom 5 to 1,300+ Clusters: Declarative Scaling for Private Kubernetes
06mSponsored Keynote: What I Learned Running 10,000+ Kubernetes Clusters
32mDon't Start With 500 Clusters: Patterns in Scaling Multi-Cluster Kubernetes…
36m101 Ways to “Break and Recover” Kubernetes Cluster
34mScaling Kubernetes up to 1,000 Control Planes - Dario Tranchitella