Programmatically creating and managing training data
summary
Programmatic labeling replaces per-example annotation with small programs called labeling functions that output labels or abstain and a label model learns weights unsupervised to produce probabilistic labels which are used to train any downstream classifier. Examples: a chest x‑ray workflow uses 20 labeling functions → ~1% accuracy gap versus hand labels by labeling reports then training an image model, a YouTube spam pipeline reaches 94.4% accuracy and internal deployments report double‑digit lifts.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:45Current Approach: Manual Labeling
- 2:08Manual Labeling Is...
- 3:11Alternative Approach: Programmatic Labeling
- 5:35The Snorkel Pipeline
- 6:18Example Labeling Function: Spam
- 9:46Reason #1: Improved Generalization
- 11:26Reason #2: Scaling with Unlabeled Data
- 13:17Chest X-Ray Classification @
- 15:16Snorkel Tutorials
- 17:041. Write Labeling Functions (LFS)
- 19:52Train a Classifier
- 21:04Other Training Data Operations
related talks
51mHandling the problem of hand-labeled training data with data programming...
33mWeak supervision: a new paradigm for unreliable data
31mReliable Data for Large ML Models: Principles and Practices
40mData labelling to scale your ML and AI pipelines
33mThe Role of Software Developers in Machine Learning
30mActive Learning and Human-in-the-Loop Machine Learning
20mPragmatic Deep Learning for image labelling
20mGreenfield vs. Brownfield Data Labeling