The Explorer's Guide To Cloud Native GenAI Platform Engineering
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 17:00· Level 1 | Hall Entrance S10 | Room B·
summary
Maps a practical path from a 15‑minute Thinnest Viable Platform (Canoe/Argo CD/Backstage) to an MVP running KServe for LLM inference and an advanced platform with Envoy AI Gateway, vector DBs and observability. KServe adds adaptive token autoscaling, model and prompt caching to cut model download latency (examples: 12‑minute cold downloads avoided) and LM cache yields ~3–10% faster time‑to‑first‑token. Also shows multi‑node/model‑sharding, token‑based cost monitoring and centralized provider routing with out‑of‑the‑box autoscaling and tracing.
AI-written summary · about
chapters (13)
- 0:00Welcome to GenAI platform engineering
- 3:11Cloud native for GenAI innovation
- 5:17Thinnest viable platform with open source
- 7:03Minimal GenAI platform setup demo
- 8:08KServe for model serving
- 11:19Self-service AI with Backstage
- 12:02Envoy AI Gateway access
- 14:13Q&A
- 18:57AI gateway routing and cost demo
- 21:05LLM-as-a-service platforms
- 22:54Optimized GenAI workloads
- 26:44Enterprise KServe caching and scaling
- 30:20Summary and future
related talks
25mCloud Native GenAI Using KServe and OPEA
24mKServe Next: Advancing Generative AI Model Serving
34mKServe in Production: Scaling Generative Inference the Cloud Native Way
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes
35mServing Large Language Models using KServe
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
31mFrom Model Serving To Distributed Inference: How Llm-d Evolves AI Platforms on…
33mFast Inference, Furious Scaling: Leveraging VLLM With KServe