Serving ML Models at Scale with Seldon and Kubeflow
summary
Seldon demonstrates a Kubernetes-native serving platform that uses Helm/Ksonnet, Argo workflows and OpenShift s2i packaging to wrap models as containers, exposes REST and gRPC endpoints, and manages deployments via a Seldon Core custom resource graph. An MNIST demo runs three models (TensorFlow, scikit‑learn, R), shows one model at ~90% vs another at ~25% accuracy, and the platform’s multi-armed bandit routing quickly shifts traffic to the best performer; canary, shadow and rolling updates work without downtime.
AI-written summary · about
chapters (17)
- 0:00Start
- 0:38Overview
- 2:55Seldon-Core Goals
- 7:22Seldon-Core Machine Learning Deployment
- 8:11seldon-core architecture
- 9:12Runtime Prediction Graphs Predictive Units
- 10:15Seldon Core Complex Graphs
- 13:18Canary
- 13:59Blue-Green Deployments
- 14:22Shadow Deployments
- 15:00Seldon-Core ML Tool Agnostic
- 15:40Openshift Source-to-Image
- 16:39Wrapping python models with 21
- 17:43Wrapping Java models with 521
- 18:18Seldon Core Workflow
- 18:33External API to connect to business
- 21:00Kubeflow Components
related talks
35mAdvanced Model Inferencing Leveraging KNative, Istio & Kubeflow Serving
26mAutomated Machine Learning Performance Evaluation
42mDeploying ML at Scale with Kubernetes and TFX
24mFrom Model to Microservice - Machine Learning at Scale- 42mDeploying ML at Scale with Kubernetes and TFX
36mAccelerating High-Performance Machine Learning at Scale i...
46mDeploying scalable Machine Learning models with Kubernetes
31mBuilding and Managing a Centralized ML Platform with Kubeflow at...