Building scalable end-to-end deep learning pipelines in the cloud
summary
Argues a serverless pipeline on AWS reduces ops and idle cost by composing AWS Lambda for short parallel tasks, AWS Batch for heavy or GPU jobs, Amazon SageMaker for managed training and AWS Fargate for long CPU workloads, all glued with AWS Step Functions. Presents benchmarks showing CPU inference can beat GPU on cost per million predictions and that Lambda is cheapest under 1 million predictions per month. Provides Terraform/code examples and a repo with deployable patterns; patterns run and scale but require benchmarking (Graviton2 often wins).
AI-written summary · about
chapters (32)
- 0:00Start
- 0:24Takeaways
- 1:02Data science process
- 2:20What is serverless
- 3:56Why serverless
- 4:29Production pipeline steps!
- 4:45Data preprocessing
- 8:55AWS Fargate
- 9:19Amazon SageMaker - Processing
- 9:52Amazon SageMaker - Training
- 10:24'Serverless' cluster comparis SageMaker
- 11:33CPU vs GPU for ML
- 12:20Inference cost - Inception V
- 14:33Price comparison - CPU
- 15:44Modular approach
- 16:40Platform-as-a-Service
- 16:50Cloud native orchestrators
- 17:35Amazon Step Functions
- 18:16SageMaker Pipelines
- 20:52Serverless approach
- 22:02AWS Batch/Fargate
- 23:08Repositories to check
- 23:32ML/DL training
- 24:19External infrastructure
- 26:03Usual AWS architecture for infe
- 27:16ML/DL inference pipeline
- 27:40How to import models
- 29:51SVM classifier - Graviton2
- 30:55Lifehacks for serverless infere
- 31:42How do you know if this is for!
- 32:19How do you know if this is NOT f
- 33:19Summary
related talks
40mBuilding scalable end-to-end deep learning pipelines in the cloud
39mBuilding model testing infrastructure and scaling AI predictions
38mBuilding Practical, Cost-Efficient GenAI Solutions Using Serverless
12mLeveraging AWS and Kubernetes for Scalable AI/ML Workloads in the Cloud
47mAdvanced Serverless Architectural Patterns on AWS
34mDeploying a Machine Learning System That Only Runs Locally
41mBuilding Serverless Machine Learning Models in the Cloud
01h 00mServerless - reality or BS - notes from the trenches