Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
summary
A framework called Ziva guides domain experts to extract concepts from representative samples (card-sorting UI) and provide five elicitation types — backup words, label perturbation, sentence simplification, concept-back keywords, and concept annotation — so data scientists get structured rationales. Pilot studies show experts produce rich taxonomies from as few as five instances; NASA-TLX reports low mental load overall though backup-words is more stressful; a rule-based model built from Ziva outputs proves reliable across multiple classes in the case study.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:10Practice & Challenge
- 2:37Overview of System
- 4:052. Attention Elicitation
- 4:29Interface Design
- 5:19Feedback loop
- 7:30Domain experts study
- 7:47Result of Evaluation (NASA TLX)
- 8:26Application: weak-supervised learning
related talks
31mWhen humans teach machines
31mKeynote Q&A – Barbara Plank
37mBuilding a Scientific Taxonomy at Scale
34mExposing Dark Data in the enterprise with custom NLP
35mHow does text become data?
33mXAI meets Natural Language Processing
36mVibing with Data -- Multi-Agent Data Modeling and Constructi...
08mLeveraging data insights to increase efficiency and innovation