When Prediction Becomes Policy: What Do Risk Models Actually Learn?
Talkat Bridging Prediction and Intervention Problems in Social Systems (Simons Institute, 2026) · 2026-01-12..01-18·
summary
Predictive risk models trained on administrative and narrative case notes in child welfare learn patterns of agency contact, surveillance intensity, and procedural decisions more than stable harm labels; analysis uses participatory LDA topic models on Wisconsin and Ontario records to surface parenting assessments, protective factors, and temporal peaks; experiments pair practitioner labels with a local LLM to classify service-plan relevance and trace trajectories, finding good accuracy on short/medium cases but reliability collapses on long, extreme cases.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:07Algorithms in public sector child welfare
- 3:40Predictive risk models in child welfare
- 5:16Ethnographic field work Wisconsin
- 7:13Algorithmic decision-making framework
- 9:03Narrative data analysis
- 12:35Narratives and agency
- 14:10Risk models and equity
- 15:48Risk and protective factors
- 20:10Risk predictions as interventions
- 23:40Translating findings to Ontario
- 26:29Evaluating LLMs for child welfare
- 27:37Key takeaways
- 28:46Q&A
related talks
55mEthical Machine Learning
42mOperationalizing predictive analytics for homelessness prevention in Los Angeles
47mKeynote: Using Data Science for Social Good: Examples, Opportunities, and Challenges
49mGetting to True Predictive Risk: Will Data Accuracy Thwart AI’s Potential?
15mOperationalizing Predictive Analytics in Education
38mTo explain or to predict?
22mHow to Detect Silent Failures in ML Models
32mData science vs real world: friends or foes