Beyond the Chatbox: What Changes When AI Runs Where You Are
summary
Argues that device constraints—not model leaderboards—should drive design, showing working examples: live translation in glasses that runs offline, Quest depth-safety and on-device OCR and captions, growers using on-phone vision without connectivity. Recommends starting from memory/KV-cache, power and backend limits, then picking runtimes like ExecuTorch runtime to split CPU/GPU load. Advises specialist models (DINO, SAM, OmniASR’s 1,600 languages) and open weights so teams can quantize, distill or anchor versions; notes a ~1B-parameter chat model fits many phones.
AI-written summary · about
chapters (14)
- 0:00On-device AI evolution
- 1:43AI in physical context
- 3:28Five reasons for on-device AI
- 6:14Real-world on-device AI examples
- 8:21Building models with device constraints
- 10:24Memory, power, latency trade-offs
- 11:50Device constraints: wearables, phones, laptops
- 13:17Orchestrated AI deployment
- 14:20ExecuTorch for multi-backend deployment
- 15:44Start with specialist models
- 16:48Open-source models for tasks
- 18:57Perception + generalist models
- 19:19Advantages of open-weight models
- 20:01Best practices for on-device AI
related talks
32mWhat I Wish I Knew Before Building On-Device AI Features
32mBuilding Once, Running Everywhere: The Future of Portable AI Applications
48mMastering the AI Shift: Adaptive Strategies for Software Evolution
41mEscaping the Cloud: High-Performance AI in your Browser
43mThe Future of AI
44mGo Beyond the Prompt: Using Context to Build Reliable AI-Assisted Android Workflows
59mKeynote: AI without the BS, for humans
01h 19mWORKSHOP 2: AI FOR EVERYONE