Scaling Test-time Inference Compute & Rise of Reasoning Models
summary
Claims that an open reasoning stack (Deepseek v3 → R1) turns a 671B base into a reasoning model using prompting with <think>/<answer> tags, two reward signals (accuracy + format + a consistency verifier) and GRPO fine‑tuning, generating 600k SFT examples plus 200k knowledge examples (≈800k total) and distilling to 32B models. Architecture tweaks include multi-latent attention, Mixture‑of‑Experts, multi‑token prediction and FB8 low‑precision; tooling shown: GSM8K data, unsloth, TRL, optional VLM, 4‑bit quantized training over ~250 epochs, and reported competitive benchmarks.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:42Train-time compute scaling
- 2:46Base model limits & reasoning rise
- 3:49Breaking down complex problems
- 5:12Test-time compute approaches
- 6:59Reasoning search techniques
- 8:23Majority voting & reward models
- 9:24Best of N & beam search
- 10:26Look-ahead & MCTS search
- 11:50DeepSeek R1 innovations
- 13:33Mixture of Experts & attention
- 16:41DeepSeek R1 training methods
- 18:26Reasoning data consistency
- 19:29GRPO & distillation
- 21:13Group Relative Policy Optimization explained
- 22:37Demo: Llama 3 with GRPO
- 27:26Training & output analysis
- 29:11Summary & future directions
- 30:13Hybrid & agentic reasoning
- 31:15Closing
related talks
31mMaking the most of test-time compute in LLMs
24mThe Rise of Reasoning Models
45mFrom Next Token Prediction to Reasoning and Beyond
01h 00mDeepSeek-R1 Thoughtology: ＜Thinking＞ about LLM Reasoning
15mOlmo-Thinking: Training a Fully Open Reasoning Model
39mUnlocking reasoning in Large language models
01h 05mAdaptive Generalization: The Role of Dynamic Evaluation in Low-Resource Settings
29mThe Evolution of LLMs: From Imitation to Reasoning