From Training to Serving: Machine Learning Models with Terraform
summary
Terraform provisions an AWS pipeline that queues JSON job messages to SQS, triggers a Java Lambda via EventBridge every minute, then starts GPU-capable ECS tasks (using the ECS run task API) to train a Flair NLP named‑entity model, upload artifacts to S3, and record status in DynamoDB. Models are served from the same ECS cluster behind an ALB; Docker images use CUDA UBI8 and can fallback to CPU. Code, Dockerfiles, and deployment steps are on GitHub and logs land in CloudWatch. DynamoDB metadata and model artifacts in S3 are central.
AI-written summary · about
chapters (9)
- 0:00Introduction and speakers
- 1:02Project goals and architecture
- 4:32Cloud infrastructure overview
- 8:10Key technologies used
- 11:59Terraform setup and deployment
- 16:36Training model pipeline
- 20:51Serving models with ECS
- 23:16Design choices and alternatives
- 25:45Future improvements and Q&A
related talks
27mStreamline your machine learning model deployment lifecycle with Terraform
42mOrchestration of Blue-Green deployment model
38mUnlocking AWS Serverless with Terraform
28mScaling Machine Learning Training and Inference
31mMaking AI work for you: Terraform engineer blueprint
43mBuilding an AWS Serverless Application with Terraform
41mBuilding Serverless Machine Learning Models in the Cloud
12mLeveraging AWS and Kubernetes for Scalable AI/ML Workloads in the Cloud