An Efficient Implementation of TensorFlow Lite for RISC-V Vectors
summary
TensorFlow Lite runs natively on a RISC‑V vector core (x280) with full Linux, LLVM and GCC toolchains and a custom quantized int8 matrix multiplication instruction that does 4x8x8→4x4 in one cycle, giving ~12× speedup over RVV int8 and pushing end-to-end networks ~4× beyond FP16 and >10× in many cases; vectorized kernels hit ~90% hot‑loop utilization (16% end‑to‑end MACs), deliver up to ~2.3 TOPS for matmul, and validate 300+ models (≈90% functionally).
AI-written summary · about
chapters (9)
- 0:00Intro
- 0:16Key Aspects of Deploying Al models on edge devices
- 4:28RISC-V Vectors for Al
- 7:25Extensions accelerate Al Computation
- 9:33SiFive Intelligence X280 Processor
- 13:33X280 for Edge Inference: TensorFlow Lite
- 16:07Edge Al Inferencing performance across various data types
- 18:38Mobile Netv1 Performance Scaling across different ISA implementations
- 21:055 SiFive Intelligence X280
related talks
13mKeynote: Taking RISC-V Intelligence Everywhere
17mExploring Static Code Generation and SIMD-Acceleration for Machine Learning on RISC-V
24mMoving to RISC-V Vector: A Practical Journey of AI Operator Optimization
14mOptimize your TensorFlow Lite models
10mSupport TVM QNN Flow on RISC-V with SIMD Computation
34mTensorFlow Lite
13mUnleashing ML Processing Power Through RISC-V Vectors: Applications, Implementation and...
12mDemo: Boosting AI on Semidynamics RISC-V Cores with Custom Tensor Instructions