Scaling ML from 0 to millions of users
summary
Compares three concrete deployment paths: single VM with deep learning AMI for quick notebooks, container clusters on ECS/EKS with CPU and GPU nodegroups (example cluster has CPU workers and p GPU workers where a GPU node runs Nvidia SMI), and fully managed Sage maker where training is “two lines of code” and deployment “one line” using the Sage maker SDK. Shows scaling to 64 GPUs for training and 16 load‑balanced instances for serving, and cites spot instances saving about 60–80% (example 64.5%).
AI-written summary · about
chapters (13)
- 0:00Intro
- 9:40Scaling alerts
- 15:33One cluster
- 16:18ECS
- 23:38Sagemaker
- 27:48Sagemaker SDK
- 28:38Training with Tensorflow
- 30:28Scaling up
- 31:48Code example
- 33:48My opinion
- 35:43Conclusion
- 36:48How to get started
- 37:38Questions
related talks
27mScaling ML to the next level: Hosting thousands of models on SageMaker
39mTrain ML models at scale with Amazon SageMaker Training
35mAccelerate deep learning in the cloud with custom ML environments
43mDesigning and Building Complex Machine Learning Projects
19mDeveloping large-scale machine learning algorithms on Amazon SageMaker
34mDeploying a Machine Learning System That Only Runs Locally
19mUsing Amazon SageMaker to develop and deploy ML models
11mEngineering Scalable ML Infrastructure for Modern Apps