KiVA: Kid-inspired Visual Analogies for Large Multimodal Models
Talkat Understanding Higher-Level Intelligence from AI, Psychology, and Neuroscience Perspectives · 2024-06-24..06-28·
summary
Introduces KiVA, a kid‑inspired benchmark with ~3,000 stimuli of common 3D objects and simple transformations, tested on 40 adults and 30 children (ages 3–7) and on GPT‑4V, LLaVA 1.5, MANTIS; adults reach ceiling, children perform significantly above chance, but models trail even three‑year‑olds, especially dropping from change detection to extrapolation. Rotation and reflection are the hardest while size and color are easy; text prompting, in‑context examples and visual tweaks give no reliable improvement. Also proposes a tougher KiVA‑adults and conjunctive curriculum.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 3:15Visual analogies in reasoning
- 6:11KiVA benchmark design
- 9:01Human vs model limits
- 11:46Prompting for performance
- 13:57Future directions
- 14:39Summary
related talks
48mGeneralization via analogy in young children and Large Models.
51mVisual Quality Assurance Using Vision-Language Models
23mAVA: Towards Agentic Video Analytics with Vision Language Models
16mKeynote: Robot Foundation Models
01h 18mComputer Vision after the Victory of Data
27mBuilding Mini Minds Our Adventures in Teaching Small LLMs to Reason
33mUsing Promptable Image Analysis for Clinical Diagnostics
41mBridging Pixels and Words: The Transformer Vision Revolution