Inference Scaling: A New Frontier for AI Capability
summary
Claims that repeated sampling produces predictable gains: coverage (pass@k) scales log-linearly with samples across four orders of magnitude and fits an exponentiated power law so small models (17M–70B) can beat frontier models given enough samples. Built systems include Large Language Monkeys (verification via unit tests yields huge coverage gains), Archon (Bayesian search mixes generators, critics, rankers to optimize cost-performance), and KernelBench for CUDA where execution+profiling verification raises fast-p results; reward models and simple selectors plateau.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 1:06Scaling laws overview
- 3:01Large language monkeys
- 6:33Theoretical scaling observations
- 13:38Verification and selection
- 25:22CodeMonkeys software engineering
- 35:36Archon architecture search
- 49:13GPU kernel design benchmarks
- 58:57Future research directions
related talks
31mMaking the most of test-time compute in LLMs
36mScaling Laws, Emergent Behaviors, and AI Democratization
26mThe scaling laws of AI Why neural networks continue to grow
41mSub-Linear Algorithms meets Large Language Models
27mLarge Language Models and the Future of AI
28mUnlocking Smarter AI with Inference-Time Scaling
13mTailor-Made Inference at Scale
44mKeynote: Tracking the Pulse of Generative AI