SHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training
summary
SHADE is a DLT-aware cache that computes per-sample importance via loss decomposition and rank-based importance scores, applies a patch sampling policy, and maintains a two-queue app cache (PQ + ghost) to keep the currently most useful samples. Implemented as small PyTorch changes and tested on a 4-node GPU cluster with HDD NFS on ImageNet and CIFAR-10 (ResNet/AlexNet/VGG), it reaches convergence 1.7–3.3× faster with 10% cache and raises hit ratio up to ~4.5× over LRU (and ~3.6× over cordial/quiver).
AI-written summary · about
chapters (10)
- 0:00Introduction to shade
- 1:46Distributed training challenge
- 3:24Limitations of local storage
- 4:28Importance sampling insights
- 6:54Challenges in caching
- 9:33Shade architecture design
- 11:37System components and usage
- 12:47Accuracy performance results
- 14:16Throughput and efficiency
- 16:04Comparison and conclusion
related talks
18mPreparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
30mQuiver: An Informed Storage Cache for Deep Learning
28mMaximize GPU Utilization for Model Training
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics
15mKV Cache: Accelerating AI Inference on Intel CPU
25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed Caching
26mEfficiently serving LLMs at scale.