Scalably Internationalizing Millions of Latent Semantic Labels
summary
Built a system called CLAWS n‑gram mapping that converts job titles into labels by matching n‑grams and trained it on 300,000 English titles → 30,000 labels, then built LEICA to auto‑derive n‑grams for other languages by translating samples and propagating labels through an n‑gram inclusion graph. French tests (5,000 titles) yield roughly 50% precision, 30% recall, English simulations hit ~85% recall but only ~60% precision; main failure mode is label over‑propagation (e.g., “developer” → software), fixable with propagation tuning and better tokenization.
AI-written summary · about
chapters (7)
- 0:00Introduction to semantic labels
- 2:25How CLAWS processes job titles
- 8:54Building the n-gram mapping manually
- 11:43Scaling internationalization challenges
- 15:54Automating n-gram deduction with LEICA
- 23:19Validating precision and recall
- 29:42Next steps and Q&A
related talks
22mHow to expand your NLP Solution to new languages
33mBulk Labelling Techniques
30mWhere there's a will, there's a way: The mad genius of LLMs as a classifiers
01h 00mHuman and machine inductive biases for compositional linguistic generalization
38mLeverage Reviews Data for Multi Label Topics Classification
31mBuilding Large-scale Localized Language Models
45mHow multilingual is your NLP model?
31mCreate text classifiers in a few hours using the open-source Label Sleuth system