How We Reduced Latency by 70% While Maintaining 99% Accuracy
summary
Phonely scales to ~100,000 AI calls per day and cuts response time by 70% latency reduction by mapping conversational flows, collecting and annotating real call data, distilling a smaller low-latency model, and hosting it on Grock while using function calling and a voice orchestration layer. P95 drops from 3s to 300ms, a custom endpointing model handles interruptions, and business tests show a 13% lift in conversions while maintaining 99% conversational-flow accuracy.
AI-written summary · about
chapters (8)
- 0:00Introduction and background
- 1:57Why voice AI matters now
- 4:23How voice AI works step by step
- 5:47Key challenges in voice AI
- 7:51Latency issues with model providers
- 8:35Business challenges in adoption
- 10:43Reducing latency and improving accuracy
- 12:32Conclusion and call to action
related talks
25mEngineering voice agents: Latency, quality, and scale
23mContact Center Voice AI: Low-Latency Intelligence Extraction from Messy Audio Streams
19mReal-Time AI: Low Latency Solutions for Interactive Applications
05mRTC @Scale, Audio ML Session
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
26mEnabling the next generation in voice technology
31mLow-Latency Solutions for Resource-Constrained Environments
15mOptimizing inference for voice models in production