Making the most of test-time compute in LLMs
summary
Claims that extra inference time lifts accuracy using parallel scaling (best-of-n ensembles) and sequential scaling (self‑verification and backtracking), with concrete techniques: budget forcing by inserting marker tokens, fine‑tuning on “think for N steps” traces, and using small critic models for verification. Experiments report big uplifts from 1,000 curated reasoning examples on an Alibaba qu 32B base, cost and overshooting issues with DeepSeek R1 thinking loops, and a surprising boost when forcing thinking in Mandarin then outputting English.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:49Test-time compute importance
- 5:04Parallel vs sequential scaling
- 8:16Uncontrolled reasoning pitfalls
- 13:27Budget forcing for thought control
- 16:43Prompting vs fine-tuning
- 19:34Critic models for reasoning
- 21:02Reasoning datasets
- 23:54Train-to-think pipeline
- 24:37Preliminary experiments
- 25:44Q&A
related talks
32mScaling Test-time Inference Compute & Rise of Reasoning Models
36mPanel Discussion
01h 08mInference Scaling: A New Frontier for AI Capability
39mUnlocking reasoning in Large language models
19mLLMs reasoning and agentic capabilities over time
31mBeyond LLMs and Maturing RAG use cases
29mThe Evolution of LLMs: From Imitation to Reasoning
01h 00mDeepSeek-R1 Thoughtology: ＜Thinking＞ about LLM Reasoning