Is that spam in my ham? A novice's inquiry into classification.
summary
Defines spam detection as a binary classification, trains a Naive Bayes model on 2,500 labeled emails (1,721 ham, 779 spam), tokenizes bodies into a bag-of-words, filters stop words with NLTK and parses HTML with lxml and the email package in Python 2.7.9, handles unseen words with Laplace smoothing, avoids floating-point underflow with Decimal or log-probabilities, and reports about 89% accuracy on held-out data with notes on false positives and simple optimizations.
AI-written summary · about
chapters (14)
- 0:00Start
- 2:58How I'll approach today's chat.
- 4:04Machine Learning in a nutshell.
- 5:26Experience.
- 6:46Classification & Machine Learning
- 7:08Spam Detection
- 8:12Naive Bayes in stats theory.
- 10:04Independent events.
- 13:09Picks category with MAP.
- 13:51Why Naive Bayes?
- 14:22Classification: Spam Detectio
- 15:34Example - Training the spam
- 21:31False Positives
- 22:44Other drawbacks with Naive E
related talks
02h 19mWorkshop: Demystifying Natural Language Processing using Python (Scikit-Learn/ Keras)
01h 24mMachine Learning with Text in scikit learn
24mHighly Scalable Discriminative Spam Filtering
42mDoes Spam have an opinion?
24mExploring Tensor Flow on 
21mFinding Spam in Short Text Fields
48mWhat fighting spammers in Beeline Anti-Fraud looks like through a Data Scientist's eyes
34mSolving Crimes with Python Using Natural Language Processing (NLP)