Understanding the RISC-V Extensions for AI
summary
RISC‑V proposals target matrix work by adding either new instructions inside vectors or new matrix state; batch.product fasttrack does eight dot‑products in parallel (e.g. 1,024 FP8→FP32 MACs per 1024‑lane instruction) and needs no new state, so it maps well to LLM decode. IME (option G advancing) stays in vector state for small edge tiles. VME adds large accumulator state and uses outer‑product/fat‑K to multiply throughput (example 4× with narrower inputs). AM is full tile state giving highest throughput for prefill but highest area and bandwidth. Data types: int8→int32, int16→int64, bfloat16, FP8 variants; fasttrack likely to appear in silicon first.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:12AI acceleration requirements
- 3:19New data types
- 5:11ISA extension overview
- 5:42Batch dot product
- 8:17Integrated matrix extensions
- 10:01Vector matrix extensions
- 11:55Attached matrix extensions
- 13:21Performance and implementation
- 16:42Q&A and conclusion
related talks
09mLightning Talk: RISC-V and AI: How an Open-Standards ISA Underpins Portability f...
11mBoosting AI on Semidynamics RISC-V Cores with Custom Tensor Instructions
23mRunning an Open Source LLM on Open Source Hardware
21mThe RISC-V Vector Cryptography Extensions
17mLLM Inference on RISC-V Embedded CPUs
21mAhead of the Plasticity Curve: AI, Biology and Technology
57mPervasive and Sustainable AI with Adaptive Computing
20mRISC-V Goes BIG