Advanced Model Inferencing Leveraging KNative, Istio & Kubeflow Serving
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 10:55· Room 17AB - San Diego Convention Center, Mezzanine Level·
summary
Kubeflow Serving integrates Istio and Knative to deliver model serving with scale-to-zero and request‑based autoscaling, built-in canary traffic splitting, and a storage initializer cache to avoid gigabyte‑scale model pulls. It standardizes prediction APIs across TensorFlow, PyTorch, ONNX and Seldon runtimes, adds transformer hooks for pre/post processing, and emits payloads as CloudEvents for logging. Explanations use Alibi anchors for tabular/text/image models and alibi‑detect flags outliers and concept drift; a Kubeflow pipeline trains then deploys default and canary revisions — demos run but one laptop demo had flaky connectivity.
AI-written summary · about
chapters (21)
- 0:00Start
- 0:34Enterprise Machine Learning
- 0:59Perception
- 4:11Community is growing!
- 5:02Production Model Serving? How hard could it be?
- 8:15KFServing Stack
- 10:09Canary Configurations
- 11:03Supported Frameworks, Components and Storage
- 11:47Inference Service Control Plane
- 12:17KFServing Deployment View
- 13:56KF Serving Data Plane vi protocol
- 14:55Canary/Pinned Examples
- 16:05Production ML Architecture
- 16:50Why Explain ML Models?
- 17:37ML Explanation Goals
- 18:13Local Black Box Explanations
- 19:22Seldon Alibi: Explain
- 19:49Anchors
- 21:24Explanation Demos
- 26:25ML Inference Analysis
- 28:14Concept Drift
related talks
44mEngaging the Kubeflow Community: Building an Enterprise-Ready AI/ML Platform
15mAI Model Inference on K8s
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
21mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
10mBuilding Machine Learning Inference Through Knative Serverless...
34mServing ML Models at Scale with Seldon and Kubeflow
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
14mKubeflow: Where ML Meets Kubernetes