Building with OpenAI: What's Ahead
summary
Demo shows a real-time API that opens a websocket two-way stateful connection to do native speech-to-speech, multimodal I/O and function/tool calls. A live app calls a show ISS tool to fetch coordinates from OpenNotify, plots charts, and lets users interrupt midresponse. Claims: GPT-4o-class models handle audio and generate speech natively, latency is ultra-low, and the assembled demo works live, proving the tooling and function-calling flow.
AI-written summary · about
chapters (8)
- 0:00AI revolution and OpenAI's mission
- 3:12Reasoning in AI models
- 4:18Multimodality unlocks new possibilities
- 5:39Introducing realtime API
- 9:31Live demo: realtime speech
- 13:32Voice-driven app building
- 19:57Future of human-computer interaction
- 21:44Call to action and tips
related talks
01h 26mBuilding voice agents with OpenAI
21mBuilding a Voice-Enabled AI Assistant With Javascript
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
32mBuilding LLM Voice Bots with Open Source Tools
45mBringing the Power of AI to Your Application
51mBuilding an AI assistant that makes phone calls
22mAI First: Applications of the Future
46mMultimodel Multimodal and Multiagent innovation with Azure AI