When Prompting Runs Out: Post-Training an Open Model
summary
Maps the cheap levers—newer base checkpoint, context engineering, RAG, tools, constrained decoding, routing—and gives the concrete exit conditions for each, then compares concrete post-training paths: SFT with a golden labeled set, RL with verifiable rewards and environment execution, and distillation, listing typical data and compute needs (hundreds-to-thousands of reward iterations; small SFT on a single GPU; RL scaling to tens/hundreds of thousands of GPUs). Uses Nemotron weights and recipes public and NemoGym environments as a runnable end-to-end example.
AI-written summary · about
chapters (11)
- 0:00Post-training open models
- 1:14Why post-training matters
- 3:20Agent success criteria
- 6:28Pre-post-training techniques
- 8:55Post-training techniques
- 10:21Fine-tuning vs reinforcement learning
- 13:15Reinforcement learning in practice
- 16:26Start small with post-training
- 17:30Building post-training environments
- 19:36NVIDIA’s agent use in chip design
- 20:19Resources and next steps
related talks
19mTraining Agentic Reasoners
42mPost-Training Small Language Models: the adventures of a practitioner
36mLanguage Model Post-Training in 2025: an Overview of Customization Options Today
43mState of GPT
30mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
32mEnd-to-End Optimizing Multi-Turn RL and High-Performance Inference in Agents with...
52mBuild a Prompt Learning Loop
24mTeaching Mistral to Reason: Post-Training with PyTorch and NVIDIA