Cloud Native Theater | Cloud Native University... Cansu Kavili Örnek and Anneli Sara Banderby
summary
Quick decision map that maps ideas to actions: containers host models (some model images reach gigabytes or terabytes from Hugging Face), GPUs require GPU scheduling and sharing strategies, scaling needs inference-aware routing and KV caching rather than simple replicas, and runtime choices include large LLM-serving runtimes recently donated to CNCF. The four entry questions are where the model lives, how it gets resources, how users access it, and how it handles load; look for sessions on multi‑tenant GPUs, from‑laptop‑to‑cluster, observability, and memory/serving patterns.
AI-written summary · about
chapters (13)
- 0:00Welcome
- 1:24AI and Kubernetes landscape
- 3:10Shadow IT to cloud AI workloads
- 4:53AI workload types in Kubernetes
- 6:37Key questions for AI deployment
- 7:19Model location and container challenges
- 8:42AI workload resource allocation
- 10:05Scaling and accessing AI models
- 11:30Predictive vs generative AI
- 12:32AI workload best practices
- 13:15Navigating KubeCon sessions
- 13:58Advice for data scientists and engineers
- 14:59Closing advice
related talks
35mPanel: Beyond the Clouds: Charting the Course for AI in the CloudNative World
27mFrom Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models
43mCloud Native - effective computing at scale
36mCloud-Native Model Serving: vLLM's Lifecycle in Kubernetes
33mCNCF TAG-Runtime: Cloud Native Open...
47mCloud Native and Sustainable, Reproducible Scientific Computing
26mCloud Native Machine Learning Systems at Day Two and Beyond
30mCloud Native AI: Harness the Power of Advanced Scheduling for High-Performance AI/ML Training