Mark Lohmeyer, Google Cloud | Red Hat Summit 2025
summary
2025 is framed as the year of inference with a push from training to multi-step reasoning models and agentic workflows that create wildly dynamic loads. Google Cloud says it ports VLLM on TPUs to cut cost-per-token and offers LLMD as an open-source routing layer that disaggregates prefill versus decode, storage tiers, and inference engines across GPUs and TPUs. It also ships accelerator-tuned features like Hyperdisk ML and promotes agent-to-agent interoperability across a growing partner ecosystem.
AI-written summary · about
chapters (12)
- 0:00Introduction and welcome
- 1:14AI workloads and market trends
- 2:39Open source and AI at Google
- 4:24Balancing cost and performance
- 5:49LLMD and distributed inference
- 6:55Enterprise bottlenecks in AI
- 8:23Ecosystem and future vision
- 10:09Customer experience and flexibility
- 11:56Openness across the stack
- 13:00Infrastructure innovations for AI
- 14:25Memory efficiency and VLLM
- 15:50Closing remarks and summary
