Performing Transcription of WebRTC Media Streams in Browser
summary
A Chrome extension captures tab audio via tabCapture and runs Whisper in WASM locally (tiny quantized Q5 model ≈50 MB, base ≈150 MB, larger models 0.5–2 GB). Streaming uses an audio worklet with shared buffer, silence detection, 8–10s chunking with small overlaps and dedupe to reduce lumpy output. It runs on older dual‑core laptops without sending audio to cloud, gives near‑offline accuracy for English but struggles with other languages, hits CPU/file‑size limits, and moves next toward WebGPU for larger models.
AI-written summary · about
chapters (8)
- 0:00Introduction and background
- 1:47Why browser transcription
- 7:49Demo of Chrome extension
- 10:00Challenges in streaming audio
- 13:58Chrome extension V3 hurdles
- 17:10Limitations and next steps
- 20:21Privacy and future plans
- 20:42Q&A
related talks
32mWebRTC Update
19mAudio Streaming - Using WebRTC for building your own Voice AIs
23mAudio and Video Manipulation
12mGoing Live from your Browser without WebRTC
24mVideo Editing in the Browser
17mThe future of communication on the web
27mWebRTC: Real-Time Communications
01h 34mReinventing Speech-to-Text Transcriptions with Go and Whisper