How We Scale up to 2k Nodes for Batch Jobs Using Cluster Autoscaler
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2023 · on Thursday, 2023-09-28· at 11:50· 2层 会议室 1 | 2F Room 1·
summary
Describes using Kubernetes and the Cluster Autoscaler at ByteDance to support bursty batch workloads that create up to 16,000 Pods and scale the cluster to 2,000 nodes. Covers concrete operational problems and fixes: reasons Cluster Autoscaler failed to scale up, causes of slow Pod creation, and why idle nodes weren’t promptly removed, plus the solutions they applied to reach 2,000-node production scale.
AI-written summary · about
related talks
24mKubernetes Batch Processing at Scale - A Scheduling Perspective
30mScaling Kubernetes to Thousands of Nodes Across Multiple Clusters, Calmly
21mMusic Matched to Different Ways to Scale in K8s, What Could Go Wrong?
42m2000 Nodes and Beyond: How We Scaled Kubernetes to 60,000-Container Clusters
35mAutoscaling Kubernetes Deployments: A (Mostly) Practical Guide
17mScaling Made Simple: Implementing Autoscaling for Optimal K8s Performance
30mAutoscaling in Kubernetes
46mAutoscaling Pods and Nodes in Kubernetes: Where to Start?