Your Biggest User Isn't Human: Designing DevTools for AI Agents
summary
Expo sees 50% of CLI commands executed by coding agents and builds a pipeline that turns agent friction into fixes: an agent-facing feedback surface yields tens of reports per day, a cheap classifier weeds spam, embeddings and a vector DB find duplicates, a frontier model maps reports to Linear issues, and background agents run fixes and verify them on cloud simulators via an MCP server. Agents also supply concrete failed-task descriptions used to generate new agent experience reports and repeatable evals.
AI-written summary · about
chapters (17)
- 0:00Designing developer tools for agents
- 1:14From SteerCode to Expo's internal harness
- 2:02Developers want to use their own agents
- 2:45How often agents run Expo CLI commands
- 3:47Expo as mobile AI infrastructure
- 5:17The agent feedback gap
- 6:04What evals can and cannot show
- 7:22Agent experience reports
- 8:29Asking agents directly
- 8:59Feedback commands and instruction surfaces
- 10:05Learning from agent feedback
- 11:44The full feedback loop
- 12:28An earlier attempt at automated collection
- 14:44Background agents that categorize reports
- 15:50Turning struggles into evals
- 16:48Three takeaways
- 17:52Closing
related talks
26mAI Agents as Your Co-Developer
35mThe Agentic AI Engineer
01h 00mKenote: AI Didn’t Replace You. It Promoted You!
30mBeyond the Hype: Using AI Coding Agents Today
07h 47mAI Agents Conference 2026 - Day 5
45mAI agents in practice - beyond the prototype
30mBuilding Tools with Tools: Modern AI-Driven Development
52mCut the Hype and Build Your Own AI Agents