Beyond Tokenization - Shaping the Future of Conversational AI
summary
Claim: conversational AI must move beyond tokenized text into real-time multimodal agents that sense gaze, body language and emotion. Company data: 3 billion devices and ~7,000 customers power deployment at scale. Built pieces and gaps: working systems show lip syncing and gaze estimation plus emotion tagging from 11lab and hume.ai prototypes, but major failures remain—audio-video latency and clock drift break interaction, echo ruins mass-market devices, and there is no benchmark or definition of “human-like.” On-device privacy vs cloud is unresolved.
AI-written summary · about
chapters (8)
- 0:00Introduction to Agora
- 1:10Beyond tokenized language
- 2:56Real-time multimodal interaction
- 4:00Hidden challenges in AI conversations
- 7:29Emotional value in AI companions
- 9:15Defining human-like interaction
- 11:23Use cases driving AI adoption
- 12:48Future of conversational AI
related talks
16mWeak signals of a vocal revolution in UIs
34m"The Past, Present, and Future of GenAI"
46mBeyond Deep Learning: The Cognitive AI Frontier
10mA Day in the Life: An Immersive Data Experience
15mMulti-Modal AI Optimization: Combining Text, Vision, and Speech Efficiently
55mConversation Is the Next Big Interface
30mLook ma, no hands! Multimodal AI-Agents in the browser
19mLLM-Enhanced Multimodal AI: Revolutionizing Audio/Video Interaction