A comparative cognition approach to evaluating AI capabilities
Talkat Understanding Higher-Level Intelligence from AI, Psychology, and Neuroscience Perspectives · 2024-06-24..06-28·
summary
A comparative cognition pipeline treats AI like another species: a Unity-based Animal-AI environment hosts ~300 adapted animal-cognition tasks for a 2019 competition with ~64 agents; winners excel at simple Y-mazes and numerosity but fail many others (best agents solve ~8% of object-permanence tests; top entry 25%), and simple heuristics sometimes beat RL. A 260,000-task O-PIAAGETS battery compares PPO and Dreamer V3 to 4–7 year olds, and a Bayesian measurement layouts model reliably recovers simulated cognitive profiles.
AI-written summary · about
chapters (13)
- 0:00Comparative cognition approach
- 1:03Intelligent behavior and Clever Hans
- 3:37Inflexible behaviors in animals and AI
- 6:48Task design challenges
- 11:27Animal AI environment
- 14:13AI versus children in object permanence
- 16:20O-PIAAGETS battery
- 21:04AI vs biological agents assumptions
- 25:45Cognitive profiles from task performance
- 32:04AI as another species?
- 38:30AI evaluation methods strengths and limits
- 40:43Biological inspiration for AI
- 41:06Q&A
