From Next Token Prediction to Reasoning and Beyond
summary
Claims concrete improvements: models move from next-token training to multi-step reasoning by combining SFT then RL for reasoning with tool-enabled rollouts. A demonstration agent actually executes browser, bash and Python steps — it downloads CSVs, runs pandas and matplotlib, and produces plots. Training scale reports cite 36 trillion tokens mostly synthetic. Benchmarks leap from single-digit to ~60–70% on SWEBench-like tasks. Model-merging and mixture-of-experts produce tuned capabilities; many examples run and verify via unit tests.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:58Prompt engineering to AI agents
- 6:25Language model training evolution
- 12:05Reasoning models rise
- 19:34Agentic benchmarks
- 23:06Synthetic data improvements
- 26:15Transformers process language
- 33:56Transformer architecture details
- 39:36Text generation to problem solving
- 40:19Future of AI impact
related talks
32mScaling Test-time Inference Compute & Rise of Reasoning Models
19mTraining Agentic Reasoners
31mMachine Reasoning and System 2 Thinking
24mThe Rise of Reasoning Models
25mFrontier AI Models: A Review of Past Nine Months
19mRedefining Intelligence: How Reasoning Is Re-Shaping AI in 2025
50mKnowledge is embedded in language neural networks but can they reason?
49mBen Hall & Barbara Fusinska