Cloud? No Thanks! I’m Gonna Run GenAI on My AI PC
summary
Claims a single AI PC runs GenAI in seconds by combining CPU GPU and NPU and using the OpenVINO Toolkit to optimize models and reduce dependencies. Demos are open source: a quantized LLaMA-style model shrinks from ~25 GB to ~5 GB and runs locally, an object detector hits ~700 FPS on CPU ~1,800+ FPS on integrated GPU and ~1,200 FPS on NPU while power drops from ~40W to ~27W to ~19W, and live demos work.
AI-written summary · about
chapters (9)
- 0:00Intro
- 1:10Conventional vs GenAI
- 3:00What is an AI PC
- 4:20AI use cases
- 5:10Software stack
- 8:40OpenVin
- 14:45Demo
- 20:15Performance Comparison
- 23:45Conclusion
related talks
16mUnleash the superpowers of the AI PC
42mAI Everywhere– accelerate your development from PC to cloud
05mAccelerate AI development from cloud to client
41mEscaping the Cloud: High-Performance AI in your Browser
11mHow did we build the Next Generation of AI PCs - Copilot+ PCs?
36mApplied AI on the edge
29mEfficient AI with Python: running AI models on CPUs
15mBuilding Gen-AI experiences using LLM and LVM