How Modern PyTorch Supercharges Multimodal Training and Inference at Luma AI
summary
Luma AI ships Ray 3 and Dream Machine (built in five months) and runs multimodal training using a unified forge config system that applies FSDP2/FSTP2 sharding, device meshes, and activation checkpointing set to 100%. Compilation uses torch.compile with regional caches stored in Redis, and inference reuses trained pieces via a lightweight model manager. Async tensor parallelism yields 10–20% runtime gains and a custom KDUNN op on H100s gives about 2–5% improvement over FlashAttention3.
AI-written summary · about
chapters (8)
- 0:00Introduction to Luma AI
- 0:43Understanding multimodality
- 1:43Scaling and efficiency
- 3:05Experimentation with PyTorch
- 5:20Forge configuration system
- 11:13Efficient distributed training
- 17:35Distributed inference
- 18:59Q&A and concluding remarks
related talks
26mAccelerating Explorations in Vision and Multimodal AI Using Pytorch...
06mKeynote: PyTorch Lightning: Powering the GenAI Revolution from Research to the...
37mLive Fireside Chat
25mInto Generative AI with PyTorch Lightning 2.0
33mAccelerating the LLM Lifecycle on the Cloud
28mScaling Multimodal Data and Reinforcement Learning with Ray
10mKeynote: How to Leverage PyTorch to Scale AI Training and Inferencing
23mHugging Face Accelerate: Making Device-Agnostic ML Training and Inference Easy...