Any [ Agent | Model | Accelerator | Cloud ]...
summary
Red Hat claims to be the leading contributor to vLLM and to have integrated community projects from the PyTorch Foundation (vLLM, LLMD, Smart Router, got LLM) into an enterprise stack used by 90% of the Fortune 500. Engineering work produces model-and-hardware specific kernels for Nvidia Blackwell GPUs and an open CPU path with Intel AMD and Arm. Built artifacts include vLLM CPU perf eval, an open-source wrapper around Guide LLM that reports throughput and latency and aims to standardize CPU inference benchmarking.
AI-written summary · about
related talks
35mThe State of vLLM
39mAccelerating Open-Source RL and Agentic Inference with vLLM
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
33mAccelerating the LLM Lifecycle on the Cloud
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
21mScaling LLM Inference with vLLM
40mScalable and Efficient LLM Serving With the VLLM Production Stack
36mCloud-Native Model Serving: vLLM's Lifecycle in Kubernetes