Expl(AI)n Like I’m 5: An Introduction To AI-Native Networking
summary
Kubernetes can host GenAI but requires new networking primitives: training uses RDMA/Infiniband and dynamic resource allocation for GPUs; serving needs token-based rate-limiting, prefix- and cache-aware routing, session affinity and failover; inference routing uses the Gateway API Inference Extension with inference pool and inference objective and an endpoint picker (external gRPC called by Envoy) to inspect request bodies and qdepth. Agentic flows use long-lived SSE/JSON‑RPC sessions, need a dual identity with an on-behalf-of token, and benefit from purpose-built AI gateways.
AI-written summary · about
chapters (17)
- 0:00AI-native networking intro
- 1:03Speakers overview
- 2:07GenAI challenges on K8s
- 4:57GenAI lifecycle & use cases
- 6:21Model training networking
- 7:25Dynamic resource allocation
- 7:46Self-hosted model serving
- 8:51Inference gateway features
- 10:34Gateway API extension
- 13:05Distributed inference with LLMD
- 14:09Agentic AI traffic
- 17:00MCP traffic & identity
- 21:18Deploy AI gateways on K8s
- 23:23Traditional GenAI tools
- 27:57AI vs traditional tools
- 29:45Agentic infrastructure steps
- 30:48Multi-cluster AI networking
related talks
26mAI Traffic Is Different: Rethinking Proxy Arc...
43mKeynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
09mKeynote: Kubernetes for GenAI: The New Operating System of the AI-First World
23mThe State of Cloud Native: The Shift Towards AI
22mAI in Kubernetes The Promise
30mBeyond Monolithic AI: Cloud Native Patterns for Dynami...
09mTransform Your Kubernetes Cluster Into a GenAI Platform: Get Ready-t...
42mEffortlessly Build, Run, Secure, and Manage Traffic for a Generative AI Application From...