Measuring AGI: Interactive Reasoning Benchmarks for ARC-AGI-3
summary
ARC Prize Foundation previews ARC‑AGI‑3: a dynamic, game-like benchmark with ~40 public training games and a hidden private evaluation set (no internet, developer unseen) that measures whether agents learn new tasks humans can do. Evaluation uses human baselines from 400+ in-person tests and reports turn-based action counts and step efficiency under the rubric skill acquisition efficiency. Games drop language and rely on four core knowledge priors (counting, basic geometry, agentness, objectness); five sandbox games launch next month, 120 targeted by Q1 2026.
AI-written summary · about
chapters (7)
- 0:00Introduction to AGI benchmarking
- 1:25ARC's mission and approach
- 3:33Defining general intelligence
- 6:22Need for interactive benchmarks
- 9:11ARC-AGI-3 sneak peek
- 13:45Evaluation and future plans
- 15:30Q&A session
related talks
33mEvaluating Your AI Agent
45mThe Curious Incident of Developing Artificial General Intelligence
19mAI's Infinite Game
52mThe Agentic Shift- Are We Reasoning, Or Just Retrieving Smarter?
45mA comparative cognition approach to evaluating AI capabilities
26mAI Agents and the Future of Human-AI Collaborative Intelligence
19mRedefining Intelligence: How Reasoning Is Re-Shaping AI in 2025
29mValuable LLM lessons learnt on Kaggle's ARC AGI Challenge