Real-time voice agents with ElevenLabs and AI SDK
summary
Demo shows a Vercel AI SDK text agent wrapped with the 11 Labs speech engine so voice works with about five lines of code. Implementation uses speech-to-text plus a callback that wraps the text agent in a callback to a streaming client and the speech engine. The engine detects interruptions, manages turn-taking, stitches context across turns, and produces a smooth ordering flow that handles mid‑sentence changes and confirms totals in real time.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:56Voice as natural interface
- 3:02Basic voice agent demo
- 4:48Unoptimized voice issues
- 6:34Eleven Labs speech engine
- 7:43Optimized voice agent demo
- 10:32How to add voice with five lines of code
- 11:35Summary
related talks
08mGive Your Chat Agent a Voice
01h 02m[Full Workshop] Building Conversational AI Agents
01h 26mBuilding voice agents with OpenAI
10mBuilding Conversational AI Agents at Scale
17mBuilding Effective Voice Agents
21mAgents at Work
01h 04m“Hello, AI?!” – Real-Time Voice Interactions with LLMs
33mDesigning Voice Agents for Real Conversations