Turning Glasses into Smart Assistants: Open Source Vision Language Models
summary
Describes retrofitting ordinary eyeglasses into low-cost smart glasses (~$30) by attaching an ESP32-CAM module (~$20) to capture images and running open-source vision-language models (PALIGEM variants). Explains using a 3B-parameter model quantized to 4-bit (~2.6–3 GB) so inference can run on a 4 GB GPU or modest CPUs, and reports live prototypes that handle low-resolution multimodal prompts, sometimes exceeding a closed model on clarity while noting limits like ESP32 heating and resolution-versus-latency tradeoffs.
AI-written summary · about
chapters (15)
- 0:00Smart glasses with AI
- 1:47AI assistance context
- 4:17Multimodal AI intro
- 7:05Visual language models
- 8:28Open vs closed models
- 10:13Open Smart Glasses
- 11:38Glasses components
- 13:05Image processing architecture
- 15:56BLM internals
- 17:22Model optimization
- 19:56Code implementation
- 22:04Real-world examples
- 25:49Conclusions
- 26:53Next steps
- 27:55Q&A
related talks
16mRVEdge-Vision: A Fully Open,Ultra-Efficient On-Device AI Platform for Smart Eyewear
30mFrom Eyewear to Silicon: RISC-V for Low-Power AI in Next-Generation Smart Glasses
40mKeynote: Open-source Multimodal AI in the Wild
14mComputer vision on edge devices
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
32mTeaching Machines to See:Vision-Language Models Made Efficient with SmolVLM
06mAcoustic-based Upper Facial Action Recognition for Smart Eyewear
20mComputer Vision at the Edge with OpenVINO