Behemoth: A Flash-centric Training Accelerator for Extreme-scale DNNs
summary
Behemoth pairs a custom flash-centric memory system with matrix-accelerator nodes to hold an extreme-scale DNN on one node and enable data-parallel training; it simplifies SSD firmware using flash-centric FMS with lightweight FTL and multi-stream allocation to treat intermediate tensors as a volatile blue stream. The platform trades retention for endurance, sustains 15+ GB/s sustained from flash, and evaluates on scaled GPT-/T5-like models via simulator-driven experiments, reporting 3.65× lower memory cost versus TPU v3 and about 2.05× higher training throughput versus a commercial SSD-attached accelerator.
AI-written summary · about
chapters (12)
- 0:00Intro
- 1:04DNN Training Process and Dataflow
- 2:04Challenges for Extreme-scale NLP Model Training
- 3:47System Overview
- 5:22Example Execution Walk-Through
- 6:21Architecting Specialized FMS for DNN Training
- 7:00Improving Bandwidth of FMS
- 8:42Improving Endurance of FMS
- 10:01Evaluation Methodology
- 10:40Memory Cost Evaluation
- 11:00Training Throughput Evaluation
- 11:42Summary
related talks
24mBLAS-on-flash: An Efficient Alternative for Large Scale ML Training and Inference?
19mML training acceleration with heterogeneous resources in Bytedance
22mCentimani: Enabling Fast AI Accelerator Selection for DNN Training with a Novel...
20mTectonic-Shift: A Composite Storage Fabric for Large-Scale ML Training
19mDeepSpeed – Efficient Training Scalability for Deep Learning Models
06mTensorPRAM: Designing a Scalable Heterogeneous Deep Learning Accelerator
26mDeepSpeed: Efficient Training Scalability for Deep Learning
24mManipulating Data in an Artificial Intelligence World