Building a Predictive Pipeline to Rapidly Detect Phishing Domains
summary
A Python pipeline ingests real-time Certificate Transparency logs and extracts features (Levenshtein distance to brands, TLD, dash/period counts, entropy) from ~800k candidate domains reduced to ~170k labeled phishing examples, trains a scikit-learn logistic regression model, and runs as Docker containers with a Jupyter notebook for reproducibility. It flags hundreds of HTTPS phishing sites, sees 1k–5k certs/sec on the feed, notes many from Let’s Encrypt, and reports test results of 1 false positive and 6 false negatives.
AI-written summary · about
chapters (9)
- 0:00Introduction and setup
- 3:11Certificate Transparency logs
- 7:43Training data and features
- 15:51Algorithm and model training
- 19:21Performance metrics
- 20:46Live demo and results
- 23:16Command-line tool demo
- 26:50Limitations and future work
- 28:39Q&A and resources
related talks
51mDeepPhish: Simulating Malicious AI
51mDeepPhish: Simulating Malicious AI
31mAI-Powered Solutions for Cybersecurity
21mMachine Learning and Cyber Security - Detecting malicious URLs in the haystack
05m[Flash talk] MUDPIPE - Malicious URL Detection for Phishing Identification and Prevention
45m Gollum: One anti-phish bot to rule them all
29mscikit-learn, machine learning and cybercrime attribution
44mPlatform Intrusion Detection with Deep Learning