A story of managing kubernetes cluster with 15k nodes & various workloads
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Thursday, 2021-12-09· at 13:15· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Describes how Ant Group scaled a Kubernetes cluster to 15,000 nodes to support mixed long-running services and offline big-data/ML training. Explains how real-traffic analysis, simulation and performance tests were used to find bottlenecks, then shows optimizations to the Kubernetes API server (reducing list/create/update/delete latencies), client-side changes (different operators), and work on etcd performance to meet SLOs.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:53Large scale K8s clusters
- 4:22How do we get there?
- 5:34Load Simulation
- 8:22SLO-based performance measuremen
- 9:48Performance tuning
- 12:23Etcd Performance
- 14:36Live upgrading
- 16:01Apiserver Intro
- 18:31Apiserver Performance
- 27:39Webhook Performance
- 29:22Controller Performance
- 31:16Apiserver related results
- 31:56Client RT Results
related talks
37mManaging Large-Scale Kubernetes Clusters Effectively and Reliably
42m2000 Nodes and Beyond: How We Scaled Kubernetes to 60,000-Container Clusters
37mManaging Large-Scale Kubernetes Clusters Effectively and Reliably
36mUnderstanding Scalability and Performance in the Kubernetes Master
37mOne Cluster to Rule Them All - Multi-Tenancy for 1k Users
36mUnderstanding Scalability and Performance in the Kubernetes Master
34mPanel: Improving and Managing Kubernetes at Scale
30mScaling Kubernetes to Thousands of Nodes Across Multiple Clusters, Calmly