Building and Managing a Centralized Kubeflow Platform at Spotify
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 11:50· Pacific Ballroom, Salon 20-22 - Marriott Marquis San Diego Marina Hotel·
summary
Spotify moves ML workflows into Kubernetes using Kubeflow and Kubeflow Pipelines, builds a centralized platform on GKE with Terraform modules driving GCP resources, kfctl → ksonnet → customized overlays, and an internal fork of TFX plus a template repo and thin CLI/scaffold. Centralization solves a shared VPC friction, offers three node pools (standard, high‑mem, GPU), Velero backups and service‑account secrets, and supports about 100 users, 15,000 runs since launch; outcome: faster model iteration and easier ops, with multi‑tenancy and Workload Identity still pending.
AI-written summary · about
chapters (17)
- 0:00Start
- 3:06280+ Squads
- 3:36Example Team Structure
- 4:12Autonomy & Decentralization
- 12:23Cons: Centralization
- 14:25Provide Common Components
- 15:06Build beyond Kubeflow
- 17:31UI Deployment Tool
- 18:08Problems
- 18:49Deployment CLI
- 23:51Kustomize Deployment
- 24:16Ideal World
- 25:16User Access
- 25:45Secure Access
- 26:14Service Accounts
- 29:11Benefits of Centralized Platform
- 30:13Lessons Learned
related talks
32mScaling Kubeflow for Multi-tenancy at Spotify
31mBuilding and Managing a Centralized ML Platform with Kubeflow at...
30mTeam Teardown: Driving Platform Adoption and Success
14mKubeflow: Where ML Meets Kubernetes
22mBuilding a MLOps Platform with Kubeflow on GKE
21mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
42mThe Challenges Managing a Kubernetes-Based Machine Learning Infrastructure
29mHow Spotify Does ML At Scale