Lightning Talk: Efficient Inference at the Edge: Performance You Need at the Lowest...
summary
Qualifies a PyTorch 2.0 workflow integrated with the Qualcomm AI Stack and an executortorch delegator that AOT compiles quantized subgraphs and offloads them to the Hexagon NPU. Examples and code live on GitHub and validate MobileNetV2, Inception, EDSR and DeepLabV3. Measured results show a MobileNetV2 path moving from ~2 seconds per image to 1500 FPS on recent silicon and DeepLabV3 at 92 FPS, demonstrating real power and latency improvements on SM8475/SM8550 devices.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:03Edge inference matters
- 3:14Qualcomm’s ML hardware
- 4:20Meta-Qualcomm NPU integration
- 6:50GitHub repo
- 7:58CPU baseline demo
- 8:39Hexagon NPU demo
- 10:44NPU acceleration gains
- 12:30Real-time inference
- 12:55Software stack future
- 13:35Q&A
related talks
14mLightning Talk: Every Millisecond Counts: The Fine-tuning Journey of an Ultra-Eff...
08mUnlocking AI at Scale for PyTorch Developers
25mOn-Device LLM Inference on Android With ExecuTorch and Qualcomm QNN
23mBringing ExecuTorch To the Next Frontiers of Edge AI
37mDeep Learning Neural Network Acceleration at the Edge
18mDeploying ML solutions with low latency in Python
25mLeveraging PyTorch for Generative AI in Distributed Edge Clouds
37mDeep Learning Neural Network Acceleration at the Edge