AI vs AI: Creating Novel Spam and Catching it with Text Generating AI
summary
Uses DistilGPT-2 fine-tuned as a classifier and a PPLM-based attribute model to generate labeled SMS; trains on UCI SMS corpus with a tiny seed of 200 samples and synthesizes 1,000 new texts (5 per seed). GPT-2 classifier outperforms Naive Bayes, Random Forest, LSTM and CNN baselines and achieves about 90% true positive rate at 1% false positive rate. Augmenting 200 real examples with generated data matches the gain from adding 800–1,000 hand-labeled messages.
AI-written summary · about
chapters (26)
- 0:00Intro
- 2:00We Are a Detective
- 2:49Signature-based Approaches
- 3:27Conventional Machine Learning Approaches
- 4:12Deep Neural Network Approaches
- 5:26Simple Approaches vs Pre-trained NLP models
- 6:36GPT (Generative Pre-trained Transformer)
- 8:25GPT2 Variants
- 8:59GPT Architecture
- 9:38GPT Heads for Downstream Tasks
- 10:57Spam Dataset and Classification Models
- 11:55Classification Performance against Test samples
- 14:19The Cost of Data Labeling
- 15:28How to enlarge training data
- 16:21Controlled Text Generation using GPT
- 17:22PPLM (Plug and Play Language Model) Overview
- 18:08PPLIM's Training Steps
- 19:15PPLIM's Optimization Steps
- 19:55Dataset for Spam Generation
- 22:25LIME (Local Interpretable Model Agnostic Explanation) Analysis
- 23:27Generated Ham Samples from Ham
- 24:34LIME Analysis
- 25:10Training with Augmented Samples
- 26:06Classification Models trained with Generated samples
- 26:54Experiment Summary
- 28:02Apply What You Have Learned Today
related talks
45mTips For Detecting The Use Of AI-Generated Text
01h 24mMachine Learning with Text in scikit learn
21mAI as an API
55mAI Showdown: The Good, The Glitchy, and The Risky
29mThe Rise of AI: A Reality Check on Generative AI
41mThe AI boom: what and why?
44mUnderstanding the abilities of AI systems: Memorization, generalization, and points in between
45mPanel: From Spam to Speech: Policing the Next Generation of "Unwanted Traffic"