Vaani:Architecting Conversational AI The end-2-end Pipeline of Modern Voice Agents
summary
Vaani converts a text agent into a real‑time voice agent using noise suppression, VAD, whisper-medium for ASR and Piper for TTS, runs ASR/LLM workloads in multiprocessing, chunks audio at 10 ms and streams LLM output with sentence‑streaming (wait for two sentences) to cut latency. A demo compares two queries about carrying sweets from Pune to Delhi: bare input returns Modak/Shreekhand/Puran poli, while adding diabetic/sugar‑free constraints yields date‑ or jaggery‑based and dry‑fruit options, showing context changes results.
AI-written summary · about
chapters (7)
- 0:00Introduction to Vaani
- 1:03Context in LLM interactions
- 4:59Vaani pipeline overview
- 7:47Pipeline components explained
- 10:16Minimal latency design
- 13:31Live demo of Vaani
- 17:44Conclusion and open-source
related talks
01h 02m[Full Workshop] Building Conversational AI Agents
22mVoice Agents That Think While They Talk
01h 58mWhat If Your Backend Could Talk? Building Real-Time AI Voice Agents
23mContact Center Voice AI: Low-Latency Intelligence Extraction from Messy Audio Streams
38mAI-Powered Conversational Interfaces
10mBuilding Conversational AI Agents at Scale
30mSystem Prompt Design for Voice Agents
01mGovAI Summit Spotlight!