Scaling Background Noise Filtration for AI Voice Agents
summary
Claims a production rollout of background-noise filtration for hospital call agents that measurably reduces transcription errors and UX failures. Uses an open-source start (DeepFilterNet) that sometimes suppresses patient voice, then a commercial Crisp model trained on 20,000 noises 10,000 voices 170+ years of audio. Engineering stacks: Python, custom CFFI/ctype bindings and session manager, Twilio, TTS/STT, Kubernetes on Azure to handle spiky loads (observed 300 to 1500 millicores per pod). Built as an embedded in the agent backend service to avoid latency; mostly effective but costly, hard to toggle, and still needs rigorous evaluation.
AI-written summary · about
chapters (28)
- 0:00AI voice agents in healthcare
- 1:29What defines an AI agent
- 2:55Agent architecture overview
- 4:19Background noise challenges
- 5:23Noise disrupts speech-to-text
- 6:44Noise filtration basics
- 8:07Voice separation with neural nets
- 9:32Testing open-source models
- 10:56Crisp commercial filtration
- 12:20Serving Crisp in agent
- 13:04Crisp impact on compute
- 14:51Azure limits for spikes
- 15:55Kubernetes automation
- 17:47K8s cluster components
- 19:12K8s deployments & config
- 20:16Moving from PaaS to K8s
- 21:20K8s migration challenges
- 22:23Phased workload migration
- 23:26Cost savings with K8s
- 24:29Setting CPU and memory limits in Kubernetes
- 25:55Monitoring pods with K9s
- 27:45CPU/memory balancing
- 28:48Demo: Crisp filtration
- 29:09Future model plans
- 30:40Q&A: Healthcare sound retention
- 31:48Q&A: Multimodal trade-offs
- 33:56Q&A: Prompts in responses
- 34:57Closing remarks
related talks
23mContact Center Voice AI: Low-Latency Intelligence Extraction from Messy Audio Streams
21mDeveloping Machine Learning Based Speech Enhancement Models for Teams and Skype
46mAI-Driven Techniques for Noise Filtration in Software Validation Logs
05mRTC @Scale, Audio ML Session
25mEngineering voice agents: Latency, quality, and scale
13mHow We Reduced Latency by 70% While Maintaining 99% Accuracy
27mUsing GenAI with Audio Data for Driving Better Customer Experience
26mEnabling the next generation in voice technology