Teaching Mistral to Reason: Post-Training with PyTorch and NVIDIA
summary
Shows a concrete pipeline that turns base models into stepwise reasoners by distillation, continual pretraining, supervised fine-tuning on synthetic traces and a final RL alignment loop; Llamaron family is released as nano 8B, super 43/49B (fits one H100) and ultra 253B, and Nvidia publishes code, verifiers and 30 million examples under a permissive license. Uses NeMo training stack and conversion to Nemo 2.0; fine‑tuned Mistral‑7B on DeepSeek R1 style data and it solves reasoning prompts the vanilla model fails.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:21Reasoning models benefits
- 1:50Reasoning models challenges
- 3:39Agentic AI use cases
- 7:12LLM post-training techniques
- 9:39Adding reasoning to models
- 11:50Llama Neotron models
- 13:18Llama Neotron distillation
- 14:22Supervised fine-tuning for reasoning
- 17:29Reinforcement learning for reasoning
- 18:52Reasoning demo
- 20:40Post-training Mistral with NeMo
- 22:28Key takeaways and Q&A
related talks
27mBuilding Mini Minds Our Adventures in Teaching Small LLMs to Reason
21mWhen Prompting Runs Out: Post-Training an Open Model
29mBuild Customize and Deploy LLMs At-Scale on Azure with NVIDIA NeMo
19mTraining Agentic Reasoners
20mEverything I Learned Training Frontier Small Models
32mScaling Test-time Inference Compute & Rise of Reasoning Models
15mOlmo-Thinking: Training a Fully Open Reasoning Model
36mPanel Discussion