Navigate the future of AI: Deploy generative models on Amazon EKS
summary
Shows how to run generative models on Amazon EKS using Kubernetes-native tooling: Ray, JupyterHub, Argo Workflows and node autoscaler Karpenter with EFA, FSx CSI for large distributed training and S3 checkpointing. Covers container image lazy loading to start jobs before 100+GB images finish downloading, GPU time-slicing and upcoming savings plans. Concrete customer wins: Anthropic cuts training costs ~40%, Quora trims costs 25% and shrinks model deployment from days to hours, Adobe raises operator ratio from 10:1 to 30:1.
AI-written summary · about
chapters (8)
- 0:00Introduction and agenda
- 1:09Why Kubernetes for ML workloads
- 10:18Amazon EKS for generative AI
- 15:38Generative AI training challenges
- 22:44Customer success stories
- 28:26ML operations on Kubernetes
- 31:38JAR stack and data strategy
- 34:10Key takeaways and resources
related talks
41mKeynote
27mGen-AI at Scale: Simplifying Orchestration of Healthcare Applications Across Multi-Cluster Environme
32mKeynote: Powering Generative AI with Kubernetes
22mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case S...
39mPowering Generative AI with Kubernetes: A Cloud Native Approach
27mGenerative AI for technology leaders
09mKeynote: Kubernetes for GenAI: The New Operating System of the AI-First World
27mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case St...