A Day in the Life of a Data Scientist. Conquer ML Lifecycle on Kubernetes
summary
End-to-end ML pipeline runs on Kubernetes using Kubeflow, TFJob CRDs, Argo workflows and Helm to spin up experiments; transfer learning on Inception v3 with a 4,000-step retrain produces a model that scores ~95% on a Fan Bingbing test image. GPU scheduling uses Azure nodes and virtual kubelet targeting Azure Container Instances to run parallel hyperparameter permutations, logs land on Azure Files and a centralized TensorBoard compares runs. Gaps remain: data versioning, Docker/YAML burden on data scientists.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:46Meet Rita Zhang
- 2:24Data Preparation
- 4:42DevOps
- 6:14Containers in Kubernetes
- 7:02Kubernetes ML solutions
- 8:09Realworld examples
- 8:57Transfer learning
- 10:39Demo
- 12:51Serving
- 14:35CoopFlow
- 20:06Hyperparameter Tuning
- 28:23Recap
- 33:58Next steps
- 36:33Outro
related talks
46mDeploying scalable Machine Learning models with Kubernetes
31mKubernetes for Machine Learning: Managing and Scaling AI Workloads
42mKubernetes for Machine Learning: Managing and Scaling AI Workloads
39mMachine Learning Made Easy on Kubernetes. DevOps for Data Scientists
51mMachine Learning using Kubeflow and Kubernetes
33mCompliant Data Management and Machine Learning on Kubernetes
54mScaling Machine Learning Pipelines in Cloud
22mThe Containerization of Machine Learning: TensorFlow: Kubernetes, and Kubeflow