Bridging the data gap between LLMs and children
Talkat Understanding Higher-Level Intelligence from AI, Psychology, and Neuroscience Perspectives · 2024-06-24..06-28·
summary
Claim: LLMs need 3–6 orders of magnitude more language tokens than children (child upper bound ~60M words; GPT-3 ~500B, Chinchilla ~1T, GPT‑4 reports ~15T). Built open multimodal corpora SAYCam (≈470 hours) and BabyView 4,000 hours target, plus CHILDES/Tiny Dialogues and BabyLM mixtures, and run tools like OpenPose, Whisper, OpenCLIP and DINOv2. Findings: training on child-directed transcripts gives worse downstream than web corpora, contrastive vision on egocentric video scales poorly versus ImageNet, multimodal grounding so far fails to close the sample gap, and careful evaluation (e.g., DevBench multimodal forced-choice) reveals task‑demand effects and response‑distribution flips as models improve.
AI-written summary · about
chapters (24)
- 0:00Bridging data gap children AI
- 1:06Sample efficiency gap in learning
- 2:58Child learning cognitive theories
- 4:43AI models as cognitive tools
- 7:51Developmental data resources
- 8:57Child-directed speech vs training data
- 11:04Egocentric child video data SAYCam
- 14:38Unsupervised learning on SAYCam
- 15:42BabyView project for child data
- 17:55Classroom learning in preschool
- 20:23Vision models BabyView SAYCam
- 23:22Model performance scaling limits
- 24:25Fair developmental psychology principles
- 26:14Task demands model vs child
- 29:13DevBench multimodal tasks
- 31:00Model vs child vocabulary responses
- 33:49Takeaways future directions
- 34:35Q&A curriculum learning failure
- 36:47Q&A curriculum learning limits
- 37:32Q&A single child data training
- 40:28Q&A sample-efficient training insights
- 42:57Q&A child vs macaque learning
- 44:45Q&A evolutionary vs individual data
- 46:52Q&A data gap conclusions
related talks
46mUnder the Hood of LLMs — all those questions you never dared ask about them
37mExpanding the Capabilities of LLMs Through Community Collaboration
57mMachine Translation of Human Languages in the Age of LLMs: Is This the End of the Language Barrier?
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
28mThe Data That Shapes Foundational LLMs
18mSmolLM: The Rise of Smol Models
51mLarge Language Models: Friend, Foe, or Otherwise
01h 18mThe Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes