Build AI Systems for Discernment, Not Approval
summary
Wharton study Shaw & Nave finds people accept AI answers over 80% of the time; Duolingo builds a keystroke copy‑typing detector with a 1% false positive model yet human reviewers accept fabricated flags at 50% false alarms accepted. A single guideline copy change improves rejections by 21%. Practical fixes include splitting labels (detect headphone pixels vs violation), structured inline markup for writing feedback, and coder agents that present plans and small PRs so reviewers deliberate rather than rubber‑stamp.
AI-written summary · about
chapters (17)
- 0:00Human-in-the-loop AI intro
- 1:04Trust in AI and surrender
- 3:32AI cheating detection case study
- 5:57Testing proctor bias with fake signals
- 7:44Fixing automation bias with guidelines
- 9:06AI-human interaction cycles
- 10:50Structured interactions for data
- 12:15Breaking down AI prompts
- 13:38Improving AI tutor feedback
- 16:05Coding agent UX pitfalls
- 17:52Coding agents as junior partners
- 18:56Design for human reasoning
- 20:18Friction for decision stakes
- 21:42Every interaction as a label
- 23:07Evaluating AI success metrics
- 23:48Engineering the human-AI loop
- 25:12Summary and call to discernment
related talks
34mBuilding Guardrails for AI Coding Systems
12mThe Comfort Trap: Getting Too Used to AI
01h 01m"Looks Good to Me": A Practical Guide to Handling AI-Generated Code
10mAI powered entomology: Lessons from millions of AI code reviews
12mDesigning AI To Scale Human Thought
23mHuman-Driven Intelligence: Where AI & Automation Fall Short
20mWill AI Replace Coding or Improve It?
22mwAIred! Learn With(out) AI