Putting Tensor Flow Models Into Production
summary
Concrete deployment steps and measured behavior appear: build a Docker image that runs TensorFlow Serving, expose gRPC and HTTP ports (500 and 501), and point a model_config to SavedModel directories. The pipeline uses exported SavedModel format and the CLI saved_model_cli to inspect signatures, submits JSON instances to the /v1/models/{name}:predict REST endpoint, and uses warm-up requests plus version policies (blue/green or gradual) to avoid p99 spikes while rolling models. Examples include Fashion MNIST and ResNet with simple latency measurements.
AI-written summary · about
chapters (18)
- 0:00Model deployment intro
- 7:13Online serving importance
- 10:26TensorFlow Serving architecture
- 16:10Continuous training pipelines
- 22:02TensorFlow Serving with Docker
- 27:48REST API and gRPC endpoints
- 33:10TensorFlow Hub models
- 33:53Training basic model
- 44:11SavedModel format
- 47:02SavedModel CLI inspection
- 51:04REST API predictions
- 55:10Pre-trained ResNet model
- 57:21Prediction latency benchmark
- 1:00:20Model warm-up importance
- 1:07:21SavedModel vs frozen graph
- 1:11:21Object detection serving
- 1:15:29Custom models for production
- 1:17:17Summary and Q&A
related talks
01h 18mPutting Tensorflow Models into Production
42mDeploying ML at Scale with Kubernetes and TFX
35mGetting Machine Learning Models Ready For Production Using Python
41mRESTful Machine Learning with Flask and TensorFlow Serving
33mDeploying and Scaling Spark ML and Tensorflow AI Models
37mSimplifying Training Deep & Serving Learning Models...using Tensorflow
42mDeep learning beyond the learning- 42mDeploying ML at Scale with Kubernetes and TFX