Overcoming I/O Bottlenecks in LLM Training with Open-Source Distributed...
summary
Alluxio implements a hierarchical distributed cache (kernel page cache → local disk pages → remote cache cluster) to cut repeated cloud reads, lower metadata latency, and boost GPU utilization. Measured results show data loading dominates 82% of job time, model deploy latency drops 15 → 3 → <1 minutes after caching, and GPU utilization rises from <20% to much higher; positional reads win for large Arrow files while streaming reads perform better for many small files.
AI-written summary · about
chapters (7)
- 0:00Alexio and data caching
- 3:10Alexio’s role in AI
- 5:01Traffic patterns and caching
- 10:22Hybrid multi-cloud caching
- 16:00Real-world caching strategies
- 21:40Future of AI caching
- 23:29Q&A
related talks
32mStreamlining Data Access for with
45mAlluxio Data Orchestration for Machine Learning
40mOpen Source Memory Speed Virtual Distributed Storage System
05mLightning Talk: How to Bring Data Locality to I/O-Intensive Workloads on Kubernetes
29mOn Data Locality in Kubernetes
01h 06mBest practices when accessing Big Data or any other data!
26mLMCache: Lower LLM Performance Costs in the Enterprise
28mOptimizing Latency-Sensitive Queries for Presto at Facebook