AI-powered infrastructure:
summary
Cloud TPU v5e claims nearly 400 teraOPS per chip, 1,600 Gbps interchip links, pods of 256 chips that can scale to tens of thousands (demo at 16,000 chips), and a $0.54 per chip starting price. It supports PyTorch, TensorFlow, and JAX via StableHLO and XLA, offers multi-slice training and multi-host inference, runs models up to two trillion parameters (single chip fits ~13B), and reports ~2x training perf/$, 2.5x inference perf/$ and up to 1.7x latency speedup.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:19What is TPU
- 1:14Landscape of AI
- 1:43Googles first TPU
- 2:24Cloud TPU V5E
- 3:34Chip count
- 4:36Software stack
- 5:34Inference stack
- 6:33High throughput
- 7:18Scale
- 8:00Training
- 9:00MultiSlice Training
- 9:38Performance Graphs
- 10:09Availability
- 10:33Consumption models
related talks
06mCloud TPU v4: Fast, flexible, and easy-to-use ML accelerators
42mCloud TPU Pods: AI Supercomputing for Large Machine Learning Problems
26mKeynote: Artificial Innovation Accelerates Artificial Intelligence
04mState-of-the-art Machine Learning research with Google tools
24mGenerative AI Chips
12mKeynote: AI Infrastructure
20mLow Power AI Edge Hardware
03mAccelerate AI adoption