Agreement and Alignment for Human-AI Collaborative Decision Making
Talkat Bridging Prediction and Intervention Problems in Social Systems (Simons Institute, 2026) · 2026-01-12..01-18·
summary
Defines a frequentist online model where each day a human and an AI exchange short probability or action messages, observe the outcome, and algorithms enforce conversation calibration (calibration conditional on the counterparty’s previous message). Provable algorithms guarantee calibration with modest T and recover Scott Aaronson–style agreement rates of about 1/epsilon^2 rounds to reach epsilon agreement. For misaligned providers, a market alignment condition (user utility lies in the convex hull of providers’ utilities) yields Nash equilibria in which the user attains the same utility as with a perfectly aligned model.
AI-written summary · about
chapters (12)
- 0:00Introduction and motivation
- 1:58Defining the problem
- 6:27Classical Bayesian approach
- 10:04Relaxing behavioral assumptions
- 13:49Conversation calibration
- 16:54Frequentist agreement protocols
- 20:43Implementing the agreement
- 25:10Theoretical convergence results
- 31:54Extensions to high dimensions
- 34:54Alignment and market competition
- 37:38Market alignment and outcomes
- 41:21Closing thoughts and Q&A
related talks
40mHuman-AI Collaboration Models for Better Decisions and Faster Workflows
31mAccuracy and Interpretability Through the Lens of Human-AI Teaming
13mHarmonising LLM Agents for Real-World Market Intelligence:[...]
20mHuman above AI
24mAI as a Teammate: Redefining Agile for Human–AI Collaboration
20mFrom Alignment to Access Control: A Unified View of GenAI...
18mCollaborative AI Engineering: One Dev, Two Dozen Agents, Zero Alignment
16mAI as intergenerational superglue