Unleashing Kubernetes Intelligence - running k8sgpt utilizing your own fine-tuned LLM
Talkat Cloud Native AI Day Europe 2024 · on Tuesday, 2024-03-19· at 10:45· Pavilion 7 | Level 7.3 | Room S03·
summary
Claims a working on‑prem pipeline that fine‑tunes an LLM and plugs it into k8sgpt so cluster diagnostics run locally (no cloud LLM calls). Uses Kubeflow pipelines to run a PyTorch fine‑tune job, produces a GGUF model artifact, deploys LocalAI in‑cluster (Helm/manifest) and points k8sgpt at that endpoint; it triages and explains kube issues in plain English. Notes practical limits: Kubeflow setup and data prep take substantial time and effort, no GPU required and many LLM formats are supported.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:04Kubernetes for AI workloads
- 2:28K8sGPT troubleshooting
- 3:50Self-contained AI environment
- 5:15Kubeflow model fine-tuning
- 6:37LocalAI in-cluster models
- 7:41Fine-tuning challenges
- 10:35Limitations and future outlook
- 11:37Conclusion and key takeaways
related talks
32mRunning and Fine-tuning Open Source LLMs on Google Kubernetes Engine
06mLightning Talk: AI-Powered Kubernetes Diagnostics With K8sGPT
01h 26mTutorial: Introduction to Kubeflow Pipelines
51mMachine Learning using Kubeflow and Kubernetes
40mUnlocking Kubernetes Intelligence: Deploying AI Foundation Mode...
21mStreamline LLM Fine-Tuning on Kubernetes with Kubeflow LLM Trainer
46mDeploying scalable Machine Learning models with Kubernetes
34mSolving Kubernetes Mysteries with K8sGPT: Your AI-Powered Troubleshooting Companion