Employing AI to extract actionable insights from qualitative user...
summary
About 10,000 in-product comments are labeled and processed by a two-layer ensemble: a binary relevance classifier followed by a multi-class topic classifier, both fine-tuned from BERT on hand-labeled data. The pipeline uses learning rates around 3e-5/5e-5 with 5 and 10 epochs and yields ~88.5% binary accuracy and just under 80% multi-class with per-topic F1s 75–84. Outputs feed a dashboard that surfaces relevant comments by topic and sentiment; current bottlenecks are manual labeling, plans include semi-automation, topic trees, and testing LSTM alternatives.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 6:00Data set
- 8:25Building the classification
- 10:19Hyperparameters
- 14:09Binary classifier
- 16:22Current use
- 17:46Next steps
related talks
31mWhen humans teach machines
43mAI enhanced UX
12mIncreasing the Speed and Accuracy of Data Labeling Through an AI Assisted Interface
38mHow Product & Data Science collaborated to halve classification costs
17mYour AI Deserves Better Data
26mAI and ML for Product Management
30mIs This Actually Getting Better? A Three-Layer Approach to Measuring AI Products
23mEffective Integration and Responsible Use of AI Platforms Into Day-to-Day Analytics EffortsGenentech