LLM Powered Agents with Kubernetes
Talkat Cloud Native + Kubernetes AI Day NA 2024 · on Tuesday, 2024-11-12· at 11:15· Salt Palace | Level 1 | Grand Ballroom A·
summary
Shows a working LLM agent stack deployed on OpenShift Kubernetes that routes queries to department-specific RAG tools (product, HR, accounts) and returns grounded answers — one demo returns a $782,000 total from the accounts tool. The pipeline uses a RAG vector DB for retrieval, MLflow tracing to log agent thoughts and tool calls, and Podman containerization with published deployment manifests in the llm agents repo. Notes latency, debugging complexity and security tradeoffs.
AI-written summary · about
chapters (8)
- 0:00Introduction to session
- 1:17Understanding LLM basics
- 4:18Introduction to RAG
- 5:57Exploring LLM agents
- 11:31Agent use case and demo
- 17:56Agent system architecture
- 20:29Conclusion and challenges
- 23:12Q&A session
related talks
10mLightning Talk: LLM-Driven Applications: Intro to LLM Agents in Kubernetes
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
30mLLMs on Autopilot: Running AI Agents on Kubernetes with Open Source Tools
41mBuilding LLM Agents – Challenges and Learnings
32mBuild your own Autonomous Agent powered by LLMs using Kubernetes
09mBuilding LLM Agents with Tool Use
50mHow to Test LLM Agents
28mChildren's Guide To LLMs on Kubernetes