Is That a Duplicate Quora Question?
summary
Uses the Quora dataset and walks through concrete features and models: TF‑IDF + SVD, fuzzy string scores, Word2Vec/GloVe sentence vectors, TF‑IDF SVD Word2Vec distances, and both classical classifiers and neural nets. Implements models in Python with Keras and XGBoost and trains on one GPU (Titan X). Reports ~81.4% with XGBoost after feature engineering and near 85% accuracy with a 174M‑parameter CNN/LSTM ensemble; state‑of‑the‑art sits around 88% and code is available.
AI-written summary · about
chapters (18)
- 0:00Duplicate question detection
- 1:11Why duplicates matter on Quora
- 2:13Quora dataset structure
- 4:42Feature engineering basics
- 6:04Fuzzy string matching
- 7:48TF-IDF and SVD for semantics
- 9:57Word2Vec embeddings
- 11:43Sentence2Vec and distance metrics
- 13:28Advanced features like WMD
- 15:23Summary of all features
- 15:44Building ML models
- 18:05Deep learning approaches
- 21:20Model architecture design
- 22:48Neural network layers
- 24:14Training the model
- 24:59Results and performance
- 25:19Kaggle competition insights
- 26:31Question and answer session
related talks
01h 38mNavigating the World of NLP and Text Analytics
38mFrom text to knowledge via ML algorithms - the Quora answer
25mXavier Amatriain, VP of Engineering, Quora @ MLconf SEA
01h 51mDeep Language Modeling for Question Answering using Keras
42mXavier Amatriain, VP of Engineering, Quora
34mClassifying Search Queries Without User Click Data
39mDeep Learning vs. Conventional Machine Learning
43mDemocratizing NLP content modeling with transfer learning using GPUs