MTIA: Meta's First Generation of AI Accelerators
summary
Meta builds a 317 mm² die in 70 nanometer technology that runs at 800 MHz with a 25 watt power budget and delivers 102 TOPS int8 or 51.2 TFLOPS fp16, organized as an 8x8 PE mesh with 128 MB on‑chip memory, up to 800 GB/s on‑chip bandwidth and 176 GB/s DRAM bandwidth. Cards use dual M.2 with eight PCIe Gen4 links (12.8 GB/s host), up to 12 cards per system (≈780 W). Software uses a PyTorch‑integrated compiler pipeline, a Knife DSL plus LLVM RISCV backend, and shows near perf/watt parity with GPUs, up to 2× on fully connected layers and up to 3× on low‑complexity models.
AI-written summary · about
chapters (10)
- 0:00Start
- 1:52Need for custom solutions
- 5:35Specification
- 6:28Chip Overview
- 7:26Processing Element (PE)
- 8:45Accelerator Board
- 9:18System Topology
- 10:40Providing developer efficiency and high performance
- 15:16Model characteristics Challenges and opportunities for MTIA
- 16:44Model performance Comparison across inference accelerators
related talks
18mCo-Designing Communication for AI Accelerators
25mHardware & Co-Design
20mLow Power AI Edge Hardware
18mCo-Designing Communication for AI Accelerators
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
07mDemo: Neural Network Acceleration on Metis, Powered by RISC-V
12mAI-powered infrastructure:
25mRace to Superintelligence: Building Low-Friction AI Infrastructure for a Heterogen...