Taming Dependency Chaos for LLM in K8s
Talkat KubeCon + CloudNativeCon China 2025 · on Wednesday, 2025-06-11· at 16:15· Level 19 | Crystal Court I·
summary
A Kubernetes controller implements a CRD-driven Dataset resource that creates PVCs and K8s Jobs to prebuild Python/C++/CUDA stacks, supports pip, conda and mamba, and can mirror Hugging Face models. First environment build takes ~10–15 minutes, subsequent mounts are ready in seconds; the system reports ~90% storage savings and ~75% download time reduction by sharing a single PVC across namespaces and doing incremental installs. The repo is https://github.com/BaizeAI/dataset and the controller runs in production.
AI-written summary · about
chapters (10)
- 0:00Dependency chaos in LLM dev
- 4:06Python and C++ conflicts
- 9:26Reusable AI environments
- 13:47Dataset CRD for dependencies
- 17:42Dataset CRD builds environments
- 20:58Models and data with Dataset CRD
- 23:27Shared datasets benefits
- 26:42Unified AI workflows
- 27:28PC package manager future
- 29:20Open source & closing
related talks
33mCompliant Data Management and Machine Learning on Kubernetes
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes
31mThe Future of Data on Kubernetes: From Database Management To AI Foundation
32mAI Beyond Autocomplete: Using LLMs To Create 1000 Kubernetes Controllers
28mProduction-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performa...
31mCluster Operations as a Service, Introducing LLM Backed Controllers.
36mBuilding a Scalable Data Science & Machine Learning cloud using Kubernetes
26mBuilding a Multi-Cluster Privately Hosted LLM Serving Platform on Ku...