Labeling Tools are Great, but What About Quality Checks?
Talkat Open AI + Data Forum @ Open Source Summit North America 2022 · on Thursday, 2022-06-23· at 14:55· Room 303/304 (Level 3), JW Marriott Austin, 110 E 2nd Street Austin, TX 78701·
summary
Argues that dataset QA improves dramatically by grouping visually similar crops so humans spot systematic mistakes; implements an open‑source browser tool that trains an unsupervised ResNet18 (contrastive pretraining), extracts mid‑level features, compresses them with k‑means clustering into visual words, and sorts images by shared visual words. Applied to Mapillary traffic signs and web‑scraped t‑shirt sets, it surfaces at least 1% error in speed limits and roughly 80% error in search‑collected t‑shirt labels, and the prototype noticeably speeds review.
AI-written summary · about
chapters (6)
- 0:00Start
- 0:11Quality Assurance in Deep Learning Data Sets
- 22:27Confusion Metrics
- 22:53Deep Learning Clustering Methods
- 23:49Summary
- 24:04Improve both the Uis and the Ai
related talks
30mEfficient data labelling with weak supervision
20mPragmatic Deep Learning for image labelling
29mGotta catch'em all recognizing sloppy work in crowdsourcing tasks
46mSW Testing: Can ML save us?
42mClean Labels for ML
16mRight data at the right time: Tools demo
20mGreenfield vs. Brownfield Data Labeling
33mBulk Labelling Techniques