Demystifying Linux NPU Subsystem: From Vision to LLM at Edge
summary
Linux lacks a unified NPU subsystem so current drivers are vendor-specific, tuned for vision, and rely on DRM for DMA-buf and fences while hiding job scheduling and memory policies in proprietary toolchains. Walkthroughs show deployment of YOLOv8 (stateless) and LLaMA3-tiny (stateful) highlighting concrete gaps: DMA mapping, SRAM partitioning, command-queue management, persistent on-chip buffers and preemptive scheduling. Edgeble fixes—quantization plus model scheduling added to SoC and PCIe NPU engine drivers—are deployed and working in real edge systems.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 1:25NPU subsystem overview
- 3:13Linux support for NPUs
- 5:03DRM and NPU limitations
- 8:55Vendor driver challenges
- 12:06LLM workloads on NPUs
- 17:18Proposed improvements
- 19:03Summary and gaps
- 19:23Q&A session
related talks
28mML on the Edge for Industry 4.0 with Arm Ethos N-78 Neural Processing Unit
21mPreview: Operating Systems
37mDeep Learning Neural Network Acceleration at the Edge
37mDeep Learning Neural Network Acceleration at the Edge
51mGoing Linux on Massive Multicore
44mReal-Time Linux on Embedded Multicore Processors
20mLow Power AI Edge Hardware
47mLinux on POWER Driver Porting, Optimization, and Test