Evolving Sigma: Adapting Kubernetes at Alibaba

on YouTube ↗36m

" summary

Sigma is a homegrown cluster manager that integrates Kubernetes to run tens of thousands of servers and legacy non-container workloads; the team adapts Kubernetes by adding an extended Kubernetes API server, custom scheduler logic, node agents and etcd-backed state, and by evolving Sigma into Sigma 3.2 while keeping Kubernetes APIs. Key outcomes include handling 32,500 transactions/sec, scaling to very large clusters, and boosting resource utilization ~30% through offline/online scheduling and autoscaling.

AI-written summary · about

" chapters (14)

1. 0:00Start
2. 3:36Sigma at scale
3. 4:43Containerization
4. 7:42Cluster management
5. 12:40Sigma and Fuxico-location
6. 14:49Old Sigma core
7. 18:26New core?
8. 18:46Kubernetes as the new core
9. 21:13Migration
10. 22:22Unified scheduler
11. 26:08In-place set controller
12. 28:02Image distribution with Dragonfly
13. 29:00Image preheat
14. 30:39Challenges

" related talks

• 41mSIG Cluster Lifecycle IntroFabrizio Pandini, Cecile Robert-Michon · KubeCon + CloudNativeCon North America 2022
• 22mKeynote: Cloud Native at Alibaba ScaleXiang Li · KubeCon + CloudNativeCon China 2019
• 38mIntro: SIG Cluster LifecycleAlexander Kanevskiy, Di Xu · KubeCon + CloudNativeCon China 2019
• 20mIntroduction to SIG Cluster LifecycleJustin Santa Barbara, Lubomir I. Ivanov · KubeCon + CloudNativeCon North America 2020 Virtual
• 28mKubernetes SIG Node Intro And Deep DiveSergey Kanzhelev, Dawn Chen, Derek Carr · KubeCon + CloudNativeCon North America 2022
• 43mIntro + Deep Dive: SIG ScalabilityMarcel Zięba, Wojciech Tyczyński · KubeCon + CloudNativeCon Europe 2022
• 38mIntro: SIG Cluster LifecycleAlexander Kanevskiy, Di Xu · KubeCon + CloudNativeCon + Open Source Summit China 2019
• 35mNo More Chaos: Audit and Inspect Kubernetes at ScaleMa Jinjing, Jie Chen · KubeCon + CloudNativeCon + Open Source Summit China 2019