Wearable Live Captions
summary
Project fits a flexible LED array inside a mask, streams phone mic to Azure Cognitive Services for speech‑to‑text, and publishes compact binary packets over the Ably MQTT broker to a microcontroller that drives an 8x28 LED matrix with a custom pixel font and scrolling engine. The browser encodes messages to avoid heavy parsing on the MCU, wiring modes and RGB/RGBW handling are configurable, scrolling speed is adjustable, and the prototype runs in real time and proves useful for a lip‑reading user; code is on GitHub.
AI-written summary · about
chapters (12)
- 0:00Start
- 9:30MICROPHONE
- 10:23THE WEB APP
- 11:26USING ACS SPEECH SDK
- 15:20AN EXAMPLE COMMAND
- 17:30THE ABLY MOTT BROKER
- 19:37THE DISPLAY DRIVER
- 22:23CREATING A PIXEL 'FONT'
- 24:41HARDWARE DIFFERENCES
- 28:39PROCESSING THE MESSAGES
- 29:40WRITING TEXT TO THE DISPLAY
- 31:39SCROLLING THE TEXT
related talks
38mWearable Live Captions
14mWearable Subtitles: Augmenting Spoken Communication with Lightweight Eyewear for All-day Captioning
45mBe My Eyes - Leveraging Generative AI to help the blind and visually impaired people
36mMaking Technology More Accessible with .NET and AI
22mBraille.js. Bringing together accessibility, JavaScript and IoT
20mNew User Interface - Part 1
24mA11Y IRL (Accessibility in real life)
09mBeyond Captions A Practical Pipeline for Live AI-Generated Sign Language Avatars