The Real Cost of a Token: The Hidden Bill of Generative AI
summary
Examines the real monetary and compute costs of tokens in generative AI, naming models and inference/serving infrastructure as the main cost drivers. Explains how tokenization, model size, latency, batch size, and deployment choices (CPU vs GPU, on‑prem vs cloud) affect the hidden bill and offers practical ways to measure and optimize token-related expenses.
AI-written summary · about
chapters (6)
- 0:00Introduction to GenAI
- 2:52Hidden token costs
- 5:19Energy use of models
- 7:25Data center impact
- 9:56Solutions to reduce impact
- 14:52Optimization and conclusion
related talks
35mGenerative AI in the real world
38mArtificial Intelligence: How Much Will It Cost You?
29mUnlocking the Future of AI: Trends, Infrastructure, and Tokenomics
01h 02mJoint Keynote Address: Scaling AI Sustainably: An Uncharted Territory
44mThe Invisible Monster: The Carbon Footprint of AI
17mGenerative AI: Keeping it real
20mFireworks AI: Go from Hype to High - Value AI
43mGenerative AI and Global Warming: co…