Reliable Migration Of 4+ Million CPU Cores of Compute to Kubernetes
summary
LinkedIn uses Temporal to automate a region-by-region migration of 2,300+ stateless services, successfully moving 3+ million CPU cores, 10 PB memory and ~3,000 replicas to Kubernetes. The system builds an application inventory, runs image-build and dry-run validation, then executes seven idempotent activities per service with staged ramps (30/70/100%) and AB tests. A supervisor hierarchy schedules and monitors hundreds of concurrent workflows, supports reliable rollbacks, and achieves the claimed migration at scale.
AI-written summary · about
chapters (13)
- 0:00Introduction to migration
- 1:52Key migration principles
- 3:38Why Temporal was chosen
- 5:02Pre-migration pipeline
- 7:26Migration workflow details
- 8:50Ramp steps and safety buffers
- 10:14Rollback workflow
- 11:37Supervisor hierarchy
- 13:26Production migration approval
- 15:34Observability and dashboards
- 16:38Engineering decisions
- 17:44Lessons learned
- 19:11Migration results
related talks
24mMigrating 2 million CPU cores to Kubernetes
45mLearning How to Pronounce Kubernetes to Production in 3 Months!
35mPerforming Infrastructure Migrations at Scale
35mKubernetes journey
01h 25mTutorial: Migration 101: From VMs to Kubernetes
14mKeynote: The Big Leap: Conquering the Legacy System and Building a Scalable Cloud Infra...
30mScaling to 5000+ Unique K8s Deployments, How We Did It
31mMoving from Mesos to Kubernetes Without Anyone Noticing