5 Steps to Deploy Cloud Native Sustainable Foundation AI Models
summary
Shows how to containerize foundation models and run Bloom (516M) and Stable Diffusion in Docker on Kubernetes, instrumented with NVIDIA DCGM exporter + Prometheus + Grafana and controlled via nvidia-smi. Demonstrates two knobs: power capping (Stable Diffusion drops from ~150W to 100W but latency rises from ~36s to ~52–62s) and DVFS frequency tuning (V100 clocks lowered below 500MHz reduce power below idle while keeping Bloom interactive: full-frequency Bloom ~120ms, tuned worst-case ≈1.4s). Provides repo, manifests, load generators, and reproducible tradeoff data.
AI-written summary · about
chapters (12)
- 0:00Sustainable Cloud AI foundation
- 2:29Energy impact of GPUs
- 8:26Power optimization theory
- 14:30Kubernetes model deployment
- 15:55Demo setup and GPU monitoring
- 20:33Bloom and Stable Diffusion workloads
- 25:22Power capping on Stable Diffusion
- 28:00GPU frequency for Bloom
- 31:37Energy vs performance trade-offs
- 36:13Frequency tuning for savings
- 38:21Future sustainability insights
- 39:19Q&A
related talks
28mComputing Large Foundational Models
40mSustainability the Container Native Way
35mSustainability Research the Cloud Native Way
43mProblems and Solutions with AI Model Deployments in k8s
27mFoundation Models: Blueprints for success
54mEconomics and Best Practices of Running AI/ML Workloads on Kubernetes
33mDeploying and Scaling Spark ML and Tensorflow AI Models
27mGreening the AI Cloud: Validating Power Models for Kubernetes Containers