Accelerating AI on Edge
summary
Models that run on-device now include Gemma 4 2B and 4B with the E2B around 1–2 GB RAM when quantized, served through the LiteRT cross‑platform stack (Android, iOS, desktop, web, IoT) with public weights on Hugging Face. Demos show local tool calling, structured JSON output and chain‑of‑thought reasoning, a gallery app and a Raspberry Pi robot prototype that works but needs tuning; benchmarks claim up to 13x acceleration and ~56 tokens/sec with mobile gains up to 35x versus Llama.
AI-written summary · about
chapters (13)
- 0:00AI on Edge introduction
- 1:17Agenda and Gemma 4 models
- 3:02Benefits of edge AI
- 4:04Gemma 4 Edge capabilities
- 5:49Agent skills demo in Gallery
- 7:11Voice agent use cases
- 9:25Custom skills in Gallery
- 10:52Deploying AI with Lite RT
- 13:39Light RT stack and cross-platform support
- 15:45NPU acceleration gains
- 16:49Performance benchmarks
- 17:30Raspberry Pi robot demo
- 18:54Summary and Q&A
related talks
16mGemma 4: Compacting Intelligence for the Edge
46mSmall AI is the next Big Thing
23m2025: On‑Device AI Goes Mainstream on Android
08mDemo: Gemma on-device with MediaPipe and TensorFlow Lite
41mBuild intelligent Android apps with Google's AI
48mThe Power of Edge AI: Revolutionizing Future of Real-Time, Autonomous Processing
55mEdgeAI - AI in the Palm of Your Hand
49mOpen Models: The Secret Weapon for Next-Generation Software