How to Eliminate the I-O Bottleneck & Continuously Feed the GPU While Training in the Cloud
summary
Claim: a caching layer called Alessio mounts S3 Azure and HDFS and streams data so GPUs never wait. Implementation uses the POSIX-like process API to load and cache datasets once, serve repeated small-file reads, and asynchronously persist outputs. Measured wins include Microsoft: 400+ tasks, >1 TB dataset across thousands of nodes cuts idle time and cloud request errors and Momo: handles 2 billion small files, 100+ TB and halves metadata traffic. The system runs in production and stabilizes storage under heavy concurrent training.
AI-written summary · about
chapters (8)
- 0:00Accelerating cloud training with Elastio
- 1:10What Elastio is
- 4:01Accelerating storage access
- 8:58Preprocessing with training
- 13:53Elastio for AI workloads
- 17:51When to use Elastio
- 20:39Elastio’s evolution for AI
- 22:47Elastio resources
related talks
27mHow to Eliminate the I/O Bottleneck and Continuously Feed the GPU While Training in the...
25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed Caching
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
28mMaximize GPU Utilization for Model Training
37mHow to Accelerate Model Training and Eliminate the I/O bottleneck for the Cloud
38mExploring Distributed Caching for Faster GPU Training with NVMe, GDS, and RDMA
24mManipulating Data in an Artificial Intelligence World
31mGotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid...