Beyond Chat: Bringing Models to the Canvas
summary
Argues that AI should live on an editable canvas instead of in chat and demonstrates a tldraw-based pipeline that sends screenshots plus full HTML to vision models (GPT‑4o, Anthropic, Gemini) to generate live UIs, games, timers and wireframes; shows a “make real” screenshot→HTML→multi‑model” flow, a node‑based TealDraw Computer that runs natural‑language actions as agentic workflows, and an editable agent‑starter with modular actions. Results are inconsistent—some assets work, many are “shitty but amazing,” with surprises like reliably autocompleted gestures and models misplacing colors or layout.
AI-written summary · about
chapters (30)
- 0:00Canvas revolution introduction
- 1:42Text to canvas evolution
- 4:55Phones canvas transition
- 7:25AI stuck in chat era
- 9:10Canvas-based AI interactions
- 10:14TL Draw demo
- 12:00Drawings to websites
- 14:07AI feedback loops
- 17:38Sketches to interfaces
- 19:00State charts to UIs
- 21:09AI games and demos
- 22:37Canvas image manipulation
- 25:04AI drawing on canvas
- 26:50Predictive canvas AI
- 27:12AI drawing with prompts
- 29:20Websites to wireframes
- 30:47Canvas app AI format
- 31:31Agentic AI workflows
- 32:13Teal Draw Computer
- 34:23Creative AI inputs
- 36:31Invention Inventor
- 39:39Community canvas experiments
- 40:43Agent Starter kit
- 41:45Modular AI agents
- 43:32AI battle simulator
- 46:24Agent customization
- 48:54Canvas vs text for AI
- 50:16Future of canvas AI
- 53:30Community showcase
- 55:00Q&A
related talks
06mInterview
19mBeyond agents: AI as a creative partner
46mBuilding AI-enhanced User Interfaces for the Web
34mMake Real: tldraw's accidental AI play
46mIntelligence as Infrastructure: What Developers Must Learn to Build with AI
47mAI-Powered Software Development From the Trenches
33mThe Dawn Of Universal Creativity
25mVirtual Tool Calling and the Future of Portable AI Toolchains