Stream Everything - Moving from Request input to Streaming input
summary
Describes a working streaming-input system called Vox Real-Time that feeds small audio frames plus autoregressive text embeddings into a causal LM, trains the model to predict silence tokens for alignment, and uses an induced lookahead (examples: 240 ms test delay, 480 ms delay gives near state‑of‑the‑art WER). Explains vector-summing of audio/video/text embeddings for multi‑modal streaming, shows early‑stop control, and reports successful real‑time transcription with reduced chunking and faster KV‑cache prefill.
AI-written summary · about
chapters (10)
- 0:00Streaming paradigm intro
- 1:50Why streaming matters for LMs
- 2:55Vox Real Time streaming ASR
- 3:59Streaming speech recognition arch
- 5:44Latency and alignment in streaming
- 7:53Streaming for multiple modalities
- 8:34Text-to-text streaming apps
- 11:03Streaming input challenges
- 13:28Faster prefill with streaming
- 14:56Closing and resources
related talks
29mFrom Quick Wins to Revolutionising Productivity & CX with GenAI
27mBuilding a Low-Latency Voice Assistant leveraging Elixir and Membrane
28mCreating fluent Frontend Experiences in an age of AI
38mUI Engineering for the AI Age
32mBuilding a ChatGPT Data Pipeline with RisingWave Stream Processor and Cassandra Vector Search
51mEvent Driven AI: Supercharging ChatGPT with RAG & LangStream
49mBuild ChatGPT RAG Data Pipeline with RisingWave Stream Processor & Vector Store
01h 14mAn Introduction to Retrieval Augmented Generation