The Quest for the Greenest LLM
summary
Independent audit of a major but unnamed LLM supplier compares a 3 billion and 120 billion parameter model across cradle‑to‑grave life cycle and shows the larger model has produced about 21,000 tCO2e so far, with electricity driving 86–90% of emissions; training used ~25,000 NVIDIA H100s (GPT‑4 used ~60,000 A100s). Findings: 314,000 m3 freshwater consumed (≈3,000 Olympic pools, ~0.5 L per conversation), 750 kg antimony‑equivalent resources, 37 g CO2 per query when amortizing training, and clear fixes—move training to cool low‑carbon regions, prefer smaller models, run open models locally, and cut needless inferences.
AI-written summary · about
chapters (15)
- 0:00AI sustainability challenges
- 1:12Speaker motivation
- 3:22AI environmental data gaps
- 4:06First LLM environmental assessment
- 5:10Study scope and model comparisons
- 6:11Carbon footprint of AI models
- 7:15Electricity use dominates emissions
- 7:59Inference vs training emissions
- 9:28Efficient inference impact
- 9:49Water use in AI operations
- 10:32Global water scarcity
- 12:20AI hardware raw materials
- 13:49Critical minerals supply
- 15:14Key takeaways
- 17:00AI's future impact
related talks
20mLLM: Efficiency and Sustainability in Practice
20mGAMING: Benchmarking Sustainable LLM Deployment on Edge Devices
14mGreener Intelligence: Reducing AI’s Carbon Footprint
01h 02mJoint Keynote Address: Scaling AI Sustainably: An Uncharted Territory
30mLLMs: Beyond the Hype - A Practical Journey to Scale
44mThe Invisible Monster: The Carbon Footprint of AI
28mAre LLMs the answer to all our problems?
45mGreen AI: Making Machine Learning Environmentally Sustainable