Lights, Camera, Inference! Video Generation as a Service With VLLM-O...
summary
Prototype stacks vLLM-Omni + LTX-2 run live text-to-video and image-to-video endpoints with ComfyUI nodes and an async video serving path, and the repo already lists support for 30+ models including day-zero LTX-2 support. Generation outputs arrive as base64 PNG frames but frequently show a slideshow effect and spaghetti limbs, limited music quality, and weak long-sequence coherence. Integration uses monkey-patching into vLLM, routing and deployment across model-sharded nodes remains unresolved, and engineering effort is needed to make it production-ready.
AI-written summary · about
chapters (9)
- 0:00VLLM Omni intro
- 2:29What is VLLM Omni
- 7:46VLLM Omni features
- 10:39Models and multimodality
- 14:13Live demo
- 16:05LTX2 model limits
- 19:58Rapid dev & community
- 20:42Current gaps
- 23:13Future of VLLM Omni
related talks
26mDeploy Your Own VideoGen Model
32mEverything you wanted to know about VLMs but were afraid to ask
45mGenerative AI for Image and Video Production
30mRoblox's Journey to Supporting Multimodality on vLLM
40mThe Marvelous Magic of Multimodal AI
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
43mLLMs in the Wild - Streaming, RAG, and Real Time GenAI at Scale
35mThe State of vLLM