Centimani: Enabling Fast AI Accelerator Selection for DNN Training with a Novel...
summary
Centimani builds a memory estimation model plus decoupled performance models and uses microbenchmarks and roofline-style measurements to pick batch size and predict DNN training throughput. It evaluates six models on four accelerators (Sova sn30, graph core IPU, CS2, Habana G2 vs Nvidia A100), reports average prediction accuracy of 93.1% single-device and 90.4% multi-device, has per-accelerator one-time measurement overhead, and outperforms baseline similarity and utilization models.
AI-written summary · about
chapters (10)
- 0:00Centimani and AI accelerator selection
- 2:33Hardware-software mismatch in DNN training
- 6:29Limitations of performance prediction
- 8:42AI accelerator analysis
- 11:57Centimani solution overview
- 13:20Memory modeling and hyperparameters
- 15:48Decoupled performance modeling
- 17:12Evaluation results and accuracy
- 20:45Overhead analysis and implications
- 21:08Conclusion and future work
related talks
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
12mBehemoth: A Flash-centric Training Accelerator for Extreme-scale DNNs
37mDeep Learning Neural Network Acceleration at the Edge
37mDeep Learning Neural Network Acceleration at the Edge
16mAccelerating Web AI on
17mUnleash All Cores: Asymmetry-Aware Scalable DNN Inference on Mobile CPUs
16mHardware acceleration for on-device Machine Learning
20mFireworks AI: Go from Hype to High - Value AI