Peer-Predictive Self-Training for Language Model Reasoning
summary
Label-free fine-tuning called Peer-Predictive Self-Training has models generate answers sequentially, treat the final aggregated answer as a hard training target, and reweight cross-entropy updates by pointwise mutual information so early responses aligned with the aggregate get smaller updates. Experiments fine-tune LoRA adapters on Gemini 2 2B, Llama (1B/3B) and Qwen 1.5B across MultiArith, MATH-500 and simultaneous equations, showing consistent accuracy gains (larger for smaller models) versus single-model self-training baselines.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 5:47Self-training pipeline
- 22:40Math reasoning results
- 34:36Theoretical intuition
- 49:49Empirical evaluation
- 51:41Q&A
related talks
42mPost-Training Small Language Models: the adventures of a practitioner
01h 38mTraining Large Language Models: Practices and Research Questions
43mState of GPT
39mUnlocking reasoning in Large language models
28mFine-Tuning Small Language Models with Python
36mPanel Discussion
20mFwdLLM: Efficient Federated Finetuning of Large Language Models with Perturbed...
22mSpecializing Small Language Models with less data