Deploying ML at Scale with Kubernetes and TFX
summary
Shows a concrete pipeline that packages a TensorFlow SavedModel (ResNet ~110 MB) to TensorFlow Serving, deploys it in Kubernetes with a LoadBalancer, and proves it works by calling REST/gRPC (example returns “military uniform” at 94%). Demonstrates model versioning and auto-reload from GCS/S3, configurable batching config for efficient GPU use, metadata endpoint for signatures, and auto-scaling HPA 1→4 driven by CPU; demo deploys and serves successfully.
AI-written summary · about
chapters (5)
- 0:00Start
- 3:54What things to take care of?
- 10:58Simple Deployments
- 23:37Inference with gRPC
- 26:25Batch inferences
related talks
42mDeploying ML at Scale with Kubernetes and TFX
39mDeploy your ML Workloads on Kubernetes with Kubeflow
22mThe Containerization of Machine Learning: TensorFlow: Kubernetes, and Kubeflow
33mDeploying and Scaling Spark ML and Tensorflow AI Models
34mServing ML Models at Scale with Seldon and Kubeflow
01h 18mPutting Tensor Flow Models Into Production
01h 18mPutting Tensorflow Models into Production
46mDeploying scalable Machine Learning models with Kubernetes