Koba: Disaggregated Data Loading for Multimodal Training at Luma AI
summary
Koba is a PyTorch-native, Lens-integrated multimodal data loader and Kojo is a Ray/Kubernetes-backed disaggregated preprocessing pool that offloads decode/tokenize/encode work off GPUs. Claims include 12x smaller response per video clip, up to 38x less media moved into trainer HBM, and peak resident memory savings of 58%, with step-time drops of ~17% (small models), ~9% (large), and case-specific 42% when tokenization dominates. Features: deterministic seeding, fast resume, caching (9x cheaper GPU processing), autoscaling and per-worker overload shedding.
AI-written summary · about
chapters (13)
- 0:00Introduction to Koba
- 1:13Luma AI and research data
- 2:18Koba’s unified data loader
- 6:39Data loading challenges
- 8:46Introducing Cog
- 9:27Cog architecture overview
- 13:06Worker routing and scaling
- 14:33Caching in Cog
- 15:57Handling overloading
- 18:49Cog beyond codecs
- 20:14Performance results
- 23:51Cog in production
- 24:54Q&A
related talks
26mEfficiently serving LLMs at scale.
27mServing PyTorch LLMs at Scale: Disaggregated Inference With Kubernetes and Llm-d
25mScaling Machine Learning Models on Kubernetes
31mBuilding and Managing a Centralized ML Platform with Kubeflow at...
29mA Scalable Platform for Training and Inference Using Kubeflow
20mEmpowering ML Workloads With Kubeflow: JAX Distributed Training and LL...
36mAccelerating High-Performance Machine Learning at Scale i...
32mMachine Learning Using Various GPU Technology With Kubeflow.