Reliable Data for Large ML Models: Principles and Practices
summary
LLM training at pretraining scale multiplies failure modes: corrupted or unlabeled corpus can waste massive GPU weeks and poison every downstream fine-tune, while human-in-the-loop labeling shifts the failure surface to subjective, dynamic tasks. Recommended concrete practices are trace lineage for data and model provenance, formal agreements between data producers and consumers about validations and recovery SLAs, and safe rollouts with canaries and snapshots so rollbacks are quick and reliable.
AI-written summary · about
chapters (9)
- 0:00ML data reliability intro
- 3:09Supervised ML problems
- 9:06How large language models differ
- 12:58LLM pre-training & fine-tuning
- 16:48LLM training failures
- 19:39Data & model lineage
- 22:49Data integrity requirements
- 27:25Safe rollouts & rollbacks
- 29:50Conclusion
related talks
30mFrom Black Box to a Known Quantity: How to Build Predictable, Reliable ML-based
38mBuilding robust machine learning systems
32mReliable Machine Learning
29mDemystifying Machine Learning in Production: Reasoning about a Large-Scale ML Platform
28mWhy we do ML model retraining wrong, and how to do better
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
14mImproving Large Language Model Performance
24mData Governance in the AI age