Building GenAI That Doesn't Cost the Earth
summary
Model training compute grows ~350,000× from 2014–2024 and data center demand may hit ~800 TWh by 2030 while Goldman Sachs projects 60% of that rise met by fossil fuels adding ~220 million tons CO2, so the session gives concrete levers: migrate workloads to AWS to cut carbon up to 99%, prefer managed services like Amazon Bedrock, SageMaker, EKS; pick smaller models, use RAG, PEFT, prompt engineering, distillation, compression and compilation; choose silicon — Trainium (×1.25–×4 efficiency) and Graviton (up to 60% less energy) — and monitor with Bedrock evaluations and CloudWatch.
AI-written summary · about
chapters (11)
- 0:00Welcome + Why Sustainable GenAI on AWS Matters
- 0:31The Energy Spike: Model Growth & Data Center Demand
- 3:46How AWS Cuts Carbon Footprint (Renewables + Efficient Data Centers)
- 5:16Start with Managed & Serverless Services (Bedrock, EKS, SageMaker)
- 7:28Model Selection: Don’t Use a Bigger LLM Than You Need
- 9:45Evaluate Models with Amazon Bedrock Evaluations
- 10:18Customize the Model: Prompting vs RAG vs Fine-Tuning vs Training from Scratch
- 12:47Choose Efficient Silicon: Trainium for Training, Graviton for Inference
- 14:36Inference Optimization: Compression, Compilation & Distillation
- 15:59Observability & Guardrails: Monitor Cost, Performance, and Responsible AI
- 17:29Recap + Continuous Improvement + Further Reading
related talks
32mOptimizing AI/ML workloads for sustainability
14mGreener Intelligence: Reducing AI’s Carbon Footprint
45mGreen AI: Making Machine Learning Environmentally Sustainable
25mBuilding a Sustainable Future
01h 02mJoint Keynote Address: Scaling AI Sustainably: An Uncharted Territory
20mSustainability architecture design for ML workloads
44mThe Invisible Monster: The Carbon Footprint of AI
30mGreen AI in Cloud Native Ecosystems: Strategies for Sustainability and Efficiency