Transforming Data Science at PepsiCo: The Kubeflow Revolution
Talkat Kubeflow Summit Europe 2024 · on Tuesday, 2024-03-19· at 14:05· Pavilion 7 | Level 7.3 | Room S02·
summary
About 15 months of work replaces isolated notebooks and ad hoc scripts with a Kubeflow-based ML platform: a single monorepo for shared code, a component library for Snowflake access and common data sources, and a small CLI called Prometheus CLI that automates one‑click auth (headless Chrome), image builds to ACR, and recurring runs. Hyperparameter search runs from simple Python dicts via a hyperparameter tuning decorator. Migration from AWS to Azure proves straightforward and onboarding non‑ML engineers to usable pipelines succeeds.
AI-written summary · about
chapters (12)
- 0:00Kubeflow at PepsiCo
- 1:03Why Kubeflow choice
- 2:51Data science workflow problems
- 6:01Kubeflow adoption challenges
- 7:48Monorepo and CLI tool
- 10:15Prometheus CLI for scientists
- 13:37Prometheus name and culture
- 15:44Enhancing Kubeflow components
- 18:36Data source and drift solutions
- 20:46Kubeflow for non-scientists
- 21:51Onboarding new users
- 23:57Q&A
related talks
35mOrganizing data engineering around Kubeflow
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
37mA Day in the Life of a Data Scientist. Conquer ML Lifecycle on Kubernetes
14mKubeflow: Where ML Meets Kubernetes
31mKubeflow - Data Science on Steroids
27mKubeflow for enabling AI Powered Drug Discovery and development...
15mKeynote: Machine Learning on Kubernetes Made Easy With Kubeflow
38mBridging the Chasm Between Research & Software Development