Building LLM Voice Bots with Open Source Tools
summary
Explains exactly how modern voice bots are built: a voice activity detector spots speech, Whisper (or alternatives) does ASR, a text LM handles intent and dialog, and TTS (11 Labs or similar) speaks; contrasts that with audio-to-audio models like GPT-4o real-time. Gives concrete latency targets (~300–500 ms), notes 13–14 demo use cases implemented, shows a live demo that upgrades subscriptions and shares full source code.
AI-written summary · about
chapters (8)
- 0:00Building LLM voice bots
- 1:46Voice interaction challenges
- 6:23Voice bot architectures
- 10:37Audio-to-text pipelines
- 13:25Audio-to-audio models
- 17:00Conversation design
- 22:38Customer support demo
- 27:46Q&A
related talks
01h 04m“Hello, AI?!” – Real-Time Voice Interactions with LLMs
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
01h 02m[Full Workshop] Building Conversational AI Agents
17mArchitecting LLM-Based Chatbots using free open-source LLMs
23mBuilding with OpenAI: What's Ahead
10mBuilding Conversational AI Agents at Scale
19mLLM-Enhanced Multimodal AI: Revolutionizing Audio/Video Interaction
01h 58mWhat If Your Backend Could Talk? Building Real-Time AI Voice Agents