Only Slightly Bent: Uber’s Kubernetes Migration Journey for Microservices

on YouTube ↗36m

" summary

Uber evaluates moving thousands of microservices from Mesos to Kubernetes, aiming to support secure service mesh and ML workloads without rewriting services. Key results: etcd sustains large values after tuning from its 50k/sec claim and API server + etcd, after parameter changes, launches 40k pods in 30s on 8k nodes. Chosen approach keeps Peloton as the scheduler and uses Peloton host manager bridged Mesos and Kubernetes, with custom CRI and Docker logging driver and a CNI+CRI port-mapping hack for host-port service discovery; work remains exploratory, not yet in production.

AI-written summary · about

" chapters (12)

1. 0:00Intro
2. 2:53Stateless Services
3. 4:25Other Compute Use Cases
4. 5:09Customized Workload Scheduler
5. 6:33Motivation to Migrate to Kubernetes
6. 8:28Evaluation of Kubernetes for Uber
7. 9:49Scalability
8. 14:44Portability
9. 19:51Broader Integration with Uber Ecosystem
10. 21:26Case Study 1 - Logging
11. 25:31Case Study 2 - Networking
12. 28:39Key Takeaways

" related talks

• 32mMigrating 2 million CPU cores to KubernetesLucy Sweet · ContainerDays Conference 2024
• 24mMigrating 2 million CPU cores to KubernetesLucy Sweet, Mathias Schwarz · KCD Denmark 2023
• 14mKeynote: Kubernetes as a Foundational Layer of InfrastructureVicki Cheung · KubeCon + CloudNativeCon China 2018
• 20mKeynote: The Challenges of Migrating 150+ Microservices to KubernetesSarah Wells · KubeCon + CloudNativeCon Europe 2018
• 36mFrom Mesos to KubernetesYin Sun · KubeCon + CloudNativeCon China 2018
• 31mKubernetes on Supporting 1 Million Bike-Taxi Drivers in IndonesiaIqbal Farabi, Giri Kuncoro · KubeCon + CloudNativeCon China 2018
• 59mJava microservices from Netflix OSS to KubernetesAleksandr Nozdrin Plotnitskiy · JPoint 2019
• 23mKubernetes and Service Mesh Upgrade Automobile Company’s IT InfrastructureChaomeng Zhang · ServiceMeshCon Europe 2021