Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training
summary
Zeus is an online optimizer that jointly tunes batch size and GPU power limit to minimize energy per target accuracy, using an exploration‑exploitation multi‑armed bandit for noisy training behavior and just-in-time energy profiling to measure throughput and power without offline fitting or hardware changes. It scans power limits (125–300W), finds cost‑optimal settings (demo picked 175W) and reports energy improvements of 15.3%–75.8% and up to 60% time reduction across recent NVIDIA GPUs on diverse DNNs.
AI-written summary · about
chapters (12)
- 0:00Introduction to Project Zeus
- 0:31Rising energy demands in AI
- 1:34Limitations of existing methods
- 2:39Energy to accuracy metric
- 4:14Inefficiency of common practices
- 5:01The efficiency trade-off
- 5:47Introducing the Zeus framework
- 7:04Zeus workflow and design
- 8:21Decoupled optimization strategy
- 10:22Evaluation results
- 12:00Live demonstration
- 14:39Conclusion and questions
related talks
13mLightning Talk: Energy-Efficient Deep Learning with PyTorch and Zeus
06mZygarde: Time-Sensitive On-Device Deep Inference and Adaptationon Intermittently-Powered Systems
15mKareus: Joint Reduction of Dynamic and Static Energy in Large Model Training
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
26mDeepSpeed: Efficient Training Scalability for Deep Learning
32mTaming the Machine: Basics of ML Models Training and Inference Optimization
14mZeRO-Offload: Democratizing Billion-Scale Model Training
01h 12m30 Golden Rules of Deep Learning Performance