A practical guide to human-in-the-loop distillation
summary
Describes a concrete workflow: prompt a large LLM, parse its text to structured output, have humans correct the outputs with human-in-the-loop distillation, then train a task-specific model via transfer learning. Cites tools and numbers: uses Spacey and Prodigy, uses GPT-3.5/4 for annotation, produces a 400 MB model at ~2,000 words/sec and six megabyte models hitting ~16,000 words/sec, reports ~10x speedups and ~15 hours of data development to beat the LLM baseline.
AI-written summary · about
chapters (8)
- 0:00Human-in-the-loop distillation intro
- 3:38Black-box AI challenges
- 7:33Task-specific distillation workflow
- 12:31Prototype to production principles
- 18:18Optimizing human feedback
- 23:54Refactoring AI systems
- 30:37Simpler models vs end-to-end
- 33:23AI dev & privacy best practices
related talks
30mLearning to wield large language models
20mLLM: Efficiency and Sustainability in Practice
29mBeyond Solo: Enhancing LLMs through Collaborative Dialogue with GAN-Inspired Approaches
24mHarnessing the Power of LLMs in Advanced Analytics
29mLocal LLM fine-tuning: a practical example
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
27mProductionizing Open Source LLMs
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs