What If Your Backend Could Talk? Building Real-Time AI Voice Agents
summary
Build a real-time AI voice agent end-to-end using streaming audio, speech-to-speech pipelines, language understanding, dialogue logic and spoken response generation. The hands-on lab covers latency and multi-turn constraints, orchestrating voice + reasoning components with an open-source framework, and practical techniques to achieve natural, low-latency conversational agents; Python and API basics required.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 4:35Voice AI concepts
- 12:50AI agent challenges AWS
- 18:14Cascaded vs real-time
- 25:44Workshop setup
- 34:23First agent Nova Sonic
- 42:13Weather agent test
- 52:29Scraping RAG retrieval
- 1:01:56AWS demo agent
- 1:18:39Guardrails limits
- 1:25:01MCP tool interaction
- 1:39:47RAG augmented gen
- 1:56:09Summary
related talks
01h 02m[Full Workshop] Building Conversational AI Agents
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
27mHow to Build Your First Real Time Voice Agent in Python...
22mVoice Agents That Think While They Talk
29mAndroid Gets a Voice: Building Real-Time Agents
32mBuilding LLM Voice Bots with Open Source Tools
23mBuilding with OpenAI: What's Ahead
01h 01mKeynote Panel: Future of AI agents