Cluster Operations as a Service, Introducing LLM Backed Controllers.
summary
A prototype controller called LMNities controller uses LLM inference (GPT‑3.5 via OpenAI API) and fine‑tuned models (TinyLlama, Llama 2) to turn natural language into kubectl + YAML actions, writing manifests and running kubectl apply. Training uses a 35k‑row kubectl dataset from Hugging Face, fine‑tuned with exal on eight A10 GPUs, cutting training loss from ~0.5 to ~0.09. Demos include an automatic CronJob that deletes pods, a cluster audit CRD and an 80% success rate for interactive service creation.
AI-written summary · about
chapters (8)
- 0:00LLM controllers intro
- 1:06LLM myths and realities
- 5:25Kubernetes controllers demo
- 9:59Fine-tuning LLMs for K8s
- 16:24Training process and parameters
- 21:52AI in cloud native ops
- 24:51LLMNES project and future
- 26:38Q&A
related talks
43mForget Kubectl and Talk to Your Clusters: Using LLMs to Simplify Kubernetes Cluster...
09mTransform Your Kubernetes Cluster Into a GenAI Platform: Get Ready-t...
30mLLMs on Autopilot: Running AI Agents on Kubernetes with Open Source Tools
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
32mRunning and Fine-tuning Open Source LLMs on Google Kubernetes Engine
30mEfficiently Deploying and Benchmarking LLMs in Kubernetes
32mAI Beyond Autocomplete: Using LLMs To Create 1000 Kubernetes Controllers
16mStreamline LLM Fine-tuning on Kubernetes With Kubeflow LLM Trainer