Decision theoretic foundations for human-AI collaboration
Talkat Bridging Prediction and Intervention Problems in Social Systems (Simons Institute, 2026) · 2026-01-12..01-18·
summary
Introduces decision-theoretic benchmarks that quantify when pairing humans and models is actually valuable, defining a value of complementation and a misreliant benchmark to split losses into wrong overall reliance versus failure to pick the right instances. Empirical reanalyses show low complementarity in some cited studies and that much loss is discrimination not trust. Implements interventions: sorting SHAP by complementary information yields ~25% larger decision gains, and an RL‑finetuned LLM is trained to surface task‑relevant signals for medical, social‑work and moderation workflows.
AI-written summary · about
chapters (6)
- 0:00Human-AI collaboration frameworks
- 2:16Human-AI team performance challenges
- 6:24Decision-theoretic AI reliance benchmark
- 17:15Measuring information value
- 22:20Complementary information in AI design
- 27:46Summary and future research
related talks
31mAccuracy and Interpretability Through the Lens of Human-AI Teaming
40mHuman-AI Collaboration Models for Better Decisions and Faster Workflows
42mRethinking AI Alignment and Evaluation under Heterogeneous Preferences
13mHuman + Machine
31mA more human-centered approach to AI
23mOpening up the box: the complexity of human interaction with AI systems
20mHuman above AI
55mThe Role of Conventions in Adaptive Human-AI Interaction