Post-Training Small Language Models: the adventures of a practitioner
summary
Practical account of post‑training: uses TRL and DPO/GRPO workflows, applies LoRA and QLoRA for 4‑bit fine‑tuning, and runs spectrum selection or model‑merging (merge‑kit) instead of full tuning. Sees measurable gains: behavior shifts from ~100 supervised examples, language improvements need tens of thousands, won a Kaggle with Italian tweaks. Trains on 48GB GPUs (~$0.40/hr), hits "no space left" traps and ASlot conversion bugs, and deploys GGUF models via llama.cpp to run on a cheap phone successfully.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:11What is a language model
- 3:20When to fine-tune
- 5:25Training libraries overview
- 6:27Supervised fine-tuning
- 7:51Preference alignment methods
- 9:19Direct preference optimization
- 11:10Tips from practice
- 13:20Memory efficient training
- 16:00Parameter selection methods
- 18:09Model merging techniques
- 19:37Reasoning models explained
- 22:08Group relative policy optimization
- 27:06GRPO in practice
- 32:57Running models on phone
- 36:15Q&A session
related talks
32mFine-tune LLMs from Kaggle Models using (Q)LoRA
28mFine-Tuning Small Language Models with Python
30mLearning to wield large language models
01h 38mTraining Large Language Models: Practices and Research Questions
39mCollect Data to Fine-Tune Your Large Language Model
31mBuilding Large-scale Localized Language Models
22mTraining a LLaMA in your Backyard: Fine-tuning Very Large...
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning