Building End-to-End Machine Learning Workflows with Kubernetes...
summary
Shows a concrete end-to-end setup: building Kubeflow Pipelines with the kfp SDK (version shown), running on Kubernetes/EKS and orchestrating Amazon SageMaker components for processing, training and deployment. Uses a subset of the 130M Amazon reviews, tokenizes to TFRecords, fine-tunes distilbert (BERT family) via SageMaker training jobs, and deploys an endpoint that returns sensible ratings (example queries map to 5 and 1). Demo logs show a processing job ~137s and a training job ~1000s, and the pipeline artifacts are viewable in the Kubeflow UI.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:05Agenda
- 1:47Why Machine Learning with Containers
- 2:44Why Machine Learning with Kubernetes
- 3:37What is Kubernetes
- 4:39Machine Learning Pipelines
- 6:57Pipeline Overview
- 7:41Creating the Pipeline
- 9:07Sample Code
- 16:44Managed Services
- 17:33Amazon Sagemaker
- 18:40CubeVlog
- 19:35Second Demo
- 30:38Summary
- 32:32Questions
related talks
51mMachine Learning using Kubeflow and Kubernetes
46mAccelerating Machine Learning DevOps with Kubeflow
12mLeveraging AWS and Kubernetes for Scalable AI/ML Workloads in the Cloud
39mDeploy your ML Workloads on Kubernetes with Kubeflow
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
54mScaling Machine Learning Pipelines in Cloud
29mML Engineering Kubeflow
36mMachine learning pipeline Kubeflow, let's start