Create the next generation of user experiences with text generation
summary
Demo implements a voice-first bookkeeping app for Vision Pro that avoids typing by combining App Intents + Siri and an LLM to generate intent parameters as JSON. Uses Q1-series models (1.8B and 7B tested, Q1 1.5 chosen), TogetherAI $25 free credit, temperature 0.1 and max_tokens 128, chatML prompt format and function calling to query local DB. Prompt engineering expands prompt (105→362 tokens) and cuts runtime on M2 from 25.6s to 9.1s; output becomes reliable on 1.8B, but not fully production-ready without recovery and TDD.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 0:24Vision Pro experience struggles
- 3:57Demo of voice-based bookkeeping app
- 6:56Why use text generation
- 9:03Choosing LLM models
- 13:15Prompt engineering basics
- 15:39Testing different prompts
- 18:21Cost and performance trade-offs
- 20:03Optimizing prompt length
- 22:51Structured prompt formats
- 24:17Returning clean JSON
- 26:48Handling intent names
- 27:53Date handling challenges
- 29:18Function calling explained
- 31:43Key techniques summary
- 32:48AI reliability considerations
- 34:09Next-gen UX implementation
- 35:33Q&A session
related talks
34mUsing Appsmith for building GenerativeAI apps
50mSpeak To Me: Voice App Conversation Practices
17mBuilding Effective Voice Agents
47mBuilding Voice-First Android Apps
27mReinventing Customer Experience with Voice and Generative AI
13mStepWrite: Adaptive Planning for Speech-Driven Text Generation
10mA Day in the Life: An Immersive Data Experience
55mDesigning for speech