Lightning Talk: Roche's Kubeflow Story: Large Models in Seconds, Costs Cut, Securit...
summary
Roche runs Kubeflow endpoints that saw 10–15 minute cold starts for 13–26 GB GPU images; adopting Karpenter EC2 node class with prebuilt EBS snapshot plus Bottlerocket mutable data volume lets nodes launch with images already on disk. A Python scanner finds inference images, EC2 snapshots enable fast restore with GP3 throughput tuned to 1 GB/s, and load tests (20K calls) show ~58% faster cold-starts, 246s faster scale-to-10, and up to ~68% cost savings while gaining immutable-OS security.
AI-written summary · about
chapters (9)
- 0:00Roche’s Kubeflow journey
- 1:45Cold start challenge
- 3:07Traditional Linux slowdowns
- 4:52Bottlerocket for faster provisioning
- 6:14Prefetching Docker images
- 7:16Storage throughput for cold starts
- 8:17Immutable infrastructure security
- 9:20Results: 60% faster cold starts
- 10:00Key takeaways
related talks
14mLightning Talk: Leveraging Kubeflow for Next-Generation AI Healthcare Solutions
29mA Scalable Platform for Training and Inference Using Kubeflow
34mSupercharge Kubeflow Performance on GPU Clusters
35mFrom Minutes to Seconds: How we reduced ML models cold start time by 90% on Kubernetes
42mKubeflow From the End User’s Perspective: The Good, The Bad, and The Ugly
27mKubeflow for enabling AI Powered Drug Discovery and development...
51mMachine Learning using Kubeflow and Kubernetes
07mLightning Talk: Slash LLM Cold-Start Times by Pre-distributing GPU...