Leveraging LLMs to build supervised datasets suitable for smaller models
summary
Claims that LLMs can produce training data fast enough to let simpler models compete, then shows what actually changed: a salary extractor (logistic regression running on CPU) starts from 20,000 manual examples, gets a test corpus of 3,000 + 6,000 additional texts, and achieves a slight F‑score gain with fewer false positives after an LLM‑based QA preannotation pipeline using QT large plus filtering and a small‑model ensemble. Skills NER uses preannotation, postprocessing and tools like Label Studio, Len Studio, BLM, and outlines and guidance with mrol 7B; automation helps but never fully replaces human review.
AI-written summary · about
chapters (7)
- 0:00LLMs for dataset creation
- 3:34Automate dataset creation
- 7:29Salary extraction with LLMs
- 10:37Pre-annotating resumé skills
- 17:45Evaluating LLM outputs
- 21:42LLM annotation limitations
- 22:47Q&A
related talks
33mDesigning LLM Applications for Massive Datasets
45mHalf hour of labeling power: Can we beat GPT?
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
25mWhen Your Dataset Has Blind Spots LLM-Based Data Augmentation
29mGenerative-AI: Usecase-Specific Evaluation of LLM-powered Applications
29mAutomating Privacy Reviews with LLMs: Challenges & Opportunities
22mIntegrating LLMs with Traditional Data Analysis
15mLLMs for Data-Driven Decision Making in Product Development