Moving to RISC-V Vector: A Practical Journey of AI Operator Optimization
summary
Implemented MatMul, GELU and LayerNorm with SVE and RVV intrinsics on a SpacemiT K1 256‑wide RVV board and measured with perf and disassembly. Findings: vector length helps but scaling is sublinear (MatMul hits ~48% double‑precision and ~38% single‑precision of theoretical), vector‑length multiplier effects depend on matrix size, and OpenBLAS RVV lacks prefetch use. Nonlinear kernels (exp, tanh, sqrt, GELU) see RVM2 gives 2–4x speedups vs scalar; recommendation: tune ML per problem and enable RVV prefetch extensions.
AI-written summary · about
chapters (10)
- 0:00Introduction to RISC-V Vector
- 1:24Hardware and Software Setup
- 3:59Vector Length Concepts
- 7:17AI Operators Overview
- 8:23OpenBLAS Vector Optimization
- 12:07GEMM Performance Results
- 14:15Length Multiplier Impact
- 17:09Nonlinear Operators in SLEEF
- 20:03SLEEF Performance Comparison
- 23:15Future Optimization Directions
related talks
16mAccelerate HPC and AI Applications with RVV Auto-Vectorization
01h 45mPorting SIMD to RISC-V Vectors | Corrected Audio
13mKeynote: Taking RISC-V Intelligence Everywhere
22mAdvancing AI Computing With Optimized Matrix Multiplication Techniques With RISC-V CPU
12mDemo: Boosting AI on Semidynamics RISC-V Cores with Custom Tensor Instructions
01h 45mPorting SIMD to RISC-V Vectors
51m"Manual" Vectorization in 2025: When the Compiler Is Not Enough
28mAn Efficient Implementation of TensorFlow Lite for RISC-V Vectors