Greening the AI Cloud: Validating Power Models for Kubernetes Containers
summary
Claims that ML can account for up to 40% of data center energy and demonstrates an open source Kepler framework (now CNCF Sandbox) that estimates energy per process, container, pod and job by combining real sensor reads on bare metal with eBPF data collection and pre-trained regression models for VMs. Shows MLPerf inference runs, Grafana dashboards, and validation against node sensors with errors described as “very close,” and explains limits (idle power, public cloud visibility).
AI-written summary · about
chapters (14)
- 0:00Energy measurement in cloud apps
- 1:02Motivation for AI workload energy
- 5:45Kepler project overview
- 7:13Kepler architecture and power model
- 10:11Bare metal vs VM deployments
- 13:21Power model limitations
- 14:25GPU energy measurement
- 16:11MLPerf benchmark tool
- 18:19Kepler on Kubernetes
- 21:27MLPerf inference with Kepler
- 22:29Kepler Grafana dashboard
- 23:33Validating Kepler power measurements
- 24:56Summary and references
- 25:18Q&A on cloud energy
related talks
05mLightning Talk: Running Energy-Aware AI Inference on Edge Kubernetes With Kepler
35mSustainability Research the Cloud Native Way
32mOptimizing AI/ML workloads for sustainability
19mResource Management Innovation for AI Workloads in Cloud Computing
40mOptimizing AI/ML Workloads
30mGreen AI in Cloud Native Ecosystems: Strategies for Sustainability and Efficiency
45mGreen AI: Making Machine Learning Environmentally Sustainable
27mHarnessing Kubernetes for Scalable AI/ML Workloads