Deploying ML at Scale with Kubernetes and TFX
summary
Deploys a SavedModel (ResNet ~110 MB stored on GCS) with TensorFlow Serving inside Kubernetes, exposing REST on port 8501 and gRPC endpoints, serving a sample image that returns “military uniform” at 94% confidence. Demonstrates HPA-based autoscaling (CPU target, min 1 → max 4 pods), load balancer, and live model versioning with auto-pull from buckets. Shows configurable batch inference parameters, a /metadata API for signatures, and end-to-end demo on an 8-node AKS cluster.
AI-written summary · about
chapters (5)
- 0:00Start
- 3:54What things to take care of?
- 10:58Simple Deployments
- 23:37Inference with gRPC
- 26:25Batch inferences
related talks
42mDeploying ML at Scale with Kubernetes and TFX
34mServing ML Models at Scale with Seldon and Kubeflow
46mDeploying scalable Machine Learning models with Kubernetes
39mDeploy your ML Workloads on Kubernetes with Kubeflow
39mIntroducing KFServing: Serverless Model Serving on Kubernetes
22mThe Containerization of Machine Learning: TensorFlow: Kubernetes, and Kubeflow
33mDeploying and Scaling Spark ML and Tensorflow AI Models
36mAccelerating Machine Learning at Scale with