Threat Hunting with Python and Pandas
summary
JHUAPL shows a repeatable pipeline that turns raw Splunk/SQL logs into machine‑learned detections: data pulled in chunks, feature extraction in Pandas, Python, Pandas, Scikit-Learn for supervised URL classification (k‑NN, decision tree, random forest) with k-NN → Random Forest via GridSearchCV and confusion‑matrix evaluation, plus an unsupervised beacon hunt using IQR to trim outliers and K‑means. A real case filters ~7,500 domains down to 38 hosts (≈321k rows) before clustering, and code writes results back to SQL and SOAR for automation.
AI-written summary · about
chapters (24)
- 0:00Introduction and agenda
- 0:38Lab background
- 1:28Past cyber incidents
- 2:46Defensive cyber strategy
- 4:42Overcoming analysis limits
- 8:04Machine learning use case
- 9:58Data preparation
- 11:21Feature engineering
- 13:17Model building
- 14:51Training and optimization
- 17:41Model evaluation
- 19:17Refining features
- 20:31Operationalizing models
- 23:04Command and control analysis
- 24:25Beaconing detection
- 27:05Data chunking and iteration
- 30:25Statistical outlier removal
- 32:00Beaconing validation
- 34:25Advanced data clustering
- 35:50K-means implementation
- 36:55Interpreting results
- 38:45Modularity and resources
- 39:30Strategic advice
- 42:25Q&A session
related talks
24mHunting Criminals with Hybrid Analytics
43mThreat hunting in the cloud
31mThreat Hunting — Demystified
48mHunting Stealth Adversaries with Graphs & AI
31mStreamlining Threat Hunting in Cloud Environments with Jupyter
26mLooking at 4 years of web honeypot attacks: tactics, ...
28m"Cyber Security Investigations with Jupyter Notebooks"
30mTrawling for IOCs: Catching C2 in a Sea of Data