Half hour of labeling power: Can we beat GPT?
summary
Claims that LLMs are best used at development time to pre‑annotate and speed dataset creation, then fine‑tune smaller task‑specific models to match or beat few‑shot LLMs. Workflow uses Prodigy teams and Prodigy for annotation, spaCy with spaCy-llm calling GPT‑3.5 for prelabels, produces ~1,200 annotations in the session, and yields a private model <1 GB that runs ~200k words/sec. Experiments show wins with small fractions of data (1%–5%) and sometimes with ~20 examples.
AI-written summary · about
chapters (8)
- 0:00Workshop challenge intro
- 1:54Generative vs predictive NLP
- 7:15LLM pre-annotation for labeling
- 10:50Pilot annotation task
- 27:50Annotation guidelines
- 31:39Collaborative labeling results
- 40:33Training small NER model
- 43:04Key takeaways
related talks
31mLeveraging LLMs to build supervised datasets suitable for smaller models
52mUnleash the Power of Large Language Models (LLMs)
35mLeveraging LLM for text augmentation in Topic Modeling
19mBeat GPT 4 & Build the Best AI For You
37mExpanding the Capabilities of LLMs Through Community Collaboration
43mKeynote: Applied NLP in the age of Generative AI
47mOperationalization of AI/ML models and use cases for ChatGPT Q&A
24mLLMs: Powering Lightning Fast Product Innovation