Lightning Talk: Environmentally Sustainable AI via Power-Aware Batch Scheduling
summary
Demonstrates a prototype that improves performance per watt for LLM training by controlling node component power and batch scheduling: integrates with Kueue/Q and MPI operator, runs a daemonset called gopm service that queries component utilization and sets GPU power caps via vendor APIs, and uses power-aware batch scheduling to honor user-specified slowdowns (example: user allows 10% slowdown) while enforcing a cluster-wide 5 kilowatt cluster limit; implemented with Kubernetes 1.28 init container sidecars and shows jobs suspended or run to meet power constraints.
AI-written summary · about
related talks
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
12mLightning Talk: Benchmarking Your Distributed ML Training on the K8s Platform
17mGREEN: Carbon-efficient Resource Scheduling for Machine Learning Clusters
05mProject Lightning Talk: Scheduling AI Workload Among Multiple Clusters
25mGreen AI with JobRunr: Sustainable Training and Inference on the JVM
05mLightning Talk: Running Energy-Aware AI Inference on Edge Kubernetes With Kepler
37mCo-Location of CPU and GPU Workloads with High Resource Efficiency
27mGreening the AI Cloud: Validating Power Models for Kubernetes Containers