The Digital Hearth: An In-the-Wild Automated Group Speech to Text System
summary
A wooden “hearth” device records multiway conversations with an 8‑channel MEMS mic array, runs on an offline Raspberry Pi gateway powered by a large USB battery, and syncs recordings back to servers where a Rev + Google ASR dual pipeline (plus an in‑house Kaldi setup) produces transcripts, speaker embeddings and topic indexes. Deployments in libraries collect several hundred hours, show measurable ASR bias by accent, and the system generally works though noisy venues and speaker separation still cause errors.
AI-written summary · about
chapters (30)
- 0:00Introduction
- 1:20Background
- 4:04LVN
- 5:41Mott Haven
- 6:11The Conversation Box
- 7:03Max Resnick
- 9:53Cathy Kramer
- 11:24Open Heart Surgery
- 12:25Version 1 Solid Wood
- 13:22Custom PCBs
- 13:36Assembly Room
- 14:04Fake News
- 16:02Pairing
- 16:31API Calls
- 16:58WiFi
- 18:03Library
- 19:03Power
- 20:24Software Updates
- 22:29Microarray
- 22:49Interleaving
- 25:16Processing Pipeline
- 27:05Raw and Range Compression
- 27:44Transcription Services
- 29:29Analysis
- 31:47Topic Indexing
- 32:50Diuresis Speech
- 35:24Highlights
- 36:28Crosspollination
- 37:19Where to Next
- 39:03Questions
related talks
19mAudio Streaming - Using WebRTC for building your own Voice AIs
07mOk Google, What Am I Doing? Acoustic Activity Recognition Bounded by Conversational Assistant ...
25mBeyond Transcription: Building Voice AI That Understands Conversations
42mdata.bythebay.io: Shubho Sengupta,
56mSpeech Recognition: How to Build Speech-to-Text Yourself
16mThe End of Awkward AI Transcriptions
19mLLM-Enhanced Multimodal AI: Revolutionizing Audio/Video Interaction
36mVoice Enabling Mobile Apps and Custom Hardware Devices