So You Want To Run AI Agents on Kubernetes: A 101 Guide
summary
Hands-on demo that turns a Python agent + LLM into a Kubernetes microservice: a kind cluster with the example DRA driver exposes GPU resource slices, an HTTP tool server on port 9090 returns slice and claim JSON, the agent uses Claude it to diagnose that five inference replicas exceed available partitions, posts a scale change to the Kubernetes API to reduce training from 2→1, and moves the fifth inference pod from Pending to Running while recommending gVisor sandboxing and RBAC limits.
AI-written summary · about
chapters (13)
- 0:00AI agents on Kubernetes
- 1:04Kubernetes GPU handling with DRA
- 3:57GPU workload resource templates
- 6:22AI agents and tool integration
- 8:54AI agent for GPU diagnostics
- 13:56Scaling tool for AI agent
- 17:30Secure AI agents in Kubernetes
- 20:46gVisor runtime for agents
- 22:31AI agent security best practices
- 23:34AI agents in production
- 25:22CNCF projects for AI agents
- 26:46TAG workloads foundation
- 27:28Q&A on GPU slicing
related talks
23mAgents in Kubernetes 101: A Crash Course on Cloud Native AI Agents
16mAI + Kubernetes: What Beginners...
38mAI in Kubernetes: How to Get Started?
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
25mWhat We Learned Securing AI Agents at Scale on Multi-Tenant Kubernetes Clusters
09mTransform Your Kubernetes Cluster Into a GenAI Platform: Get Ready-t...
27mFrom Chatbots to Agents: Orchestrating Agentic AI on Existing Enterprise Kubernetes Infrastructure.
25mExtending Kubernetes for AI | Lessons Learned From Platform Engineering