Disaggregated Intelligence: Scaling AI Agents with Dynamo and Kubernetes
summary
Evolution from stateless LLMs to tool-augmented agents, memory and session-heavy workflows, and a concrete deployment path: Open Claw plus NeMo Tron Ultra models running behind Dynamo and Kubernetes with a Run:AI scheduler. Dynamo implements KV-aware routing and prefill/decode worker split, offloads cold history to host or S3, and claims about 4x faster first token by pinning system prompts and prefetching; the demo centers on a KV block manager, health-checked worker migration, and a Kubernetes operator for production scale.
AI-written summary · about
chapters (8)
- 0:00Agentic AI systems
- 3:50LLM usage evolution
- 8:23Agent loops in practice
- 11:09Caching in agent workflows
- 11:51Dynamo as orchestrator
- 13:17Disaggregated serving
- 15:22KV cache strategies
- 17:53Future LLM orchestration
related talks
38mFrom Chatbots To Agentic AI: Running NVIDIA Dynamo the Kubernetes Way
40mDynamo: Supporting Next-Generation AI Workloads
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
28mScaling AI Infrastructure
17mAI TechOps: Data and Infra for AI Foundation to Scale and Manage 1000s of Cohesive Agents
27mFrom Chatbots to Agents: Orchestrating Agentic AI on Existing Enterprise Kubernetes Infrastructure.
24mAI Agent Swarms Are Amazing
25mWhat We Learned Securing AI Agents at Scale on Multi-Tenant Kubernetes Clusters