To Infinite Scale and Beyond: Operating Kubernetes Past the Steady State
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 15:20· Pacific Ballroom, Salon 20-22 - Marriott Marquis San Diego Marina Hotel·
summary
Spotify runs 280 teams and 1,500 engineers deploying about 1,200 microservices many times per day, so the operational focus is on automating frequent cluster upgrades and simplifying multi-cluster deployments. Practical patterns include soak time with rapid/regular/stable release channels, using a canary cluster and automated alerts, adopting surge upgrades up to 20 nodes or creating new node pools to speed rollouts, and a deployment stack of Tugboat + Spinnaker with Compass to map services to regions and Backstage for cross-cluster observability.
AI-written summary · about
chapters (27)
- 0:00Intro
- 0:45Who is Jago
- 1:44Agenda
- 2:30Steady State
- 4:05Scaling
- 4:51Operational Challenges
- 5:10Server Lists
- 7:10Core Question
- 8:00Soak Time
- 9:45The Math is Not
- 12:45The Solution
- 13:18The Good News
- 15:50Upgrades
- 17:07Haunted Graveyard
- 19:25Automating Upgrades
- 20:49Config as Data
- 24:35Multitenant ephemeral clusters
- 25:02Challenges
- 25:57Learnings
- 26:51CICD Stack
- 27:27Compass
- 27:45Deployment UML
- 28:10Deployment Region
- 28:56Compass Abstraction
- 29:50Backstage
- 30:26Other techniques
- 31:02Building on those alternatives
related talks
14mOpen Source Ecosystem: Building Sustainable Solutions with Kubernetes
34mEmbracing Upstream Kubernetes in Web Scale Organization
03mFrom 5 to 1,300+ Clusters: Declarative Scaling for Private Kubernetes
42mMastering Kubernetes: Challenges, Strategies, and Best Practices
22m10 Kubernetes is hard!
45mHow Kubernetes Was the Secret Sauce in Our Globally Distributed Team’s Transition to Microservices
08mSponsored Keynote: Free Range Kubernetes: Feeding, Watering & Herding - Brian Goodman-Jones
46mWhy Kubernetes is Not Enough