Enabling Intelligent Media Playback on RISC-V - Running VLC With Whisper STT and Qwen T2...
summary
Runs VLC on an ESWIN EIC7702 SoC with a 40 TOPS NPU, integrating Whisper STT and Qwen T2T by porting a 7B model to the MPU and adding GGML tiny integration. Porting hits include resolving a 128 build dependencies compile chain, custom FFmpeg builds, and unpredictable MPU memory behavior. Achieves 5→10 tokens/sec translation throughput depending on memory/GPU availability. Provides hardware to external developers to accelerate riscv AI app ecosystem growth.
AI-written summary · about
chapters (6)
- 0:00Introduction and background
- 3:36RISC-V AI hardware strategy
- 7:16Choosing VLC for AI integration
- 10:08Enabling Whisper STT in VLC
- 13:23Overcoming Whisper limitations
- 15:10Demo results and next steps
related talks
17mLLM Inference on RISC-V Embedded CPUs
23mEmpower Upstream ML Frameworks on RISC-V
23mRunning an Open Source LLM on Open Source Hardware
33mTranslate tech talks using open-source models and cloud-native Wasm
57mPervasive and Sustainable AI with Adaptive Computing
31mFrom ML to LLM on device AI in the browser
13mHow Open-Source vLLM Topped the Artificial Analysis Leaderboard
30mOrchestrating Real-Time Multimodal AI Agents with Rust