Lessons Learned in Using RISC-V for Generative AI and Where We Can Go from Here

on YouTube ↗18m

" summary

Generative AI workloads hit a memory and compute wall, so Esperanto builds RISC-V based inference accelerators and software that actually run open LLMs: EToc1 ships with 32 GB LPDDR4 on EToc1, implements block 4-bit quantization to store weights and claims 4× data fetch speed with 4-bit quantization, and reports 2.4× measured end-to-end speedup so far. Early ISA-required custom SIMD uses ~40 instructions and 16 registers to load a cache line; adding RVV and a 2D tensor register drastically cuts instruction count and improves overlap of prefetch and compute.

AI-written summary · about

" chapters (8)

1. 0:00Introduction and challenges
2. 0:50Generative AI walls
3. 1:20Energy and deployment costs
4. 5:17Esperanto strategy overview
5. 5:39Software stack evolution
6. 10:27Blocking quantization
7. 10:43Dequantization kernels
8. 18:13Conclusion

" related talks

• 13mKeynote: Taking RISC-V Intelligence EverywhereCharlie Su · RISC-V Summit North America 2023
• 17mRISC-V is Ready for Powering the Era of Intelligent General ComputingCharlie Su · RISC-V Summit North America 2025
• 19mDemocratizing Inference of Open-weight Models on RISC-V Manycore Acc...R. Shaposhnik, T. Dadasheva · RISC-V Summit North America 2025
• 30mRISC-V: Enabling Open Physical AILuca Benini · RISC-V Summit Europe 2026
• 10mBoosting AI on Semidynamics RISC-V Cores with Custom Tensor InstructionsRoger Espasa · RISC-V at embedded world 2024
• 12mDemo: Boosting AI on Semidynamics RISC-V Cores with Custom Tensor InstructionsJosé María Arnau · RISC-V Summit North America 2023
• 19mAccelerating Edge AI with the Help of RISC-VFlorian Zaruba · RISC-V Summit North America 2023
• 24mMoving to RISC-V Vector: A Practical Journey of AI Operator OptimizationGuodong Xu · RISC-V Summit North America 2025