Leveraging Streaming-Based Outlier Detection and SliceLine to Stop Heavily Distributed Bot Attacks
summary
A production pipeline detects distributed layer‑7 bot attacks by computing time‑series aggregates (requests, unique UAs, IPs, sessions, countries) and flagging anomalies with streaming z‑score anomaly detection in Flink, then auto‑generates blocking rules with Sliceline via one‑hot encoding and matrix multiplication implemented in Python/Numpy (1000x speedup over R). It outputs conjunctive signatures for rule engines, blocks credential‑stuffing/scraping/carding, and stopped >3 million requests in a week from ~187k distinct IPs in a real case.
AI-written summary · about
chapters (14)
- 0:00Start
- 2:19Attacks conducted by bots
- 3:29Bot Technologies: Automated Browsers
- 4:19How to detect bots?
- 5:33Detection Example: Selenium
- 6:26Bypass Techniques Used by Bots
- 12:21Detecting Distributed Attacks
- 17:54Generating Rules Using Sliceline
- 19:08How does it really work?
- 20:53Sliceline Internals: Encoding
- 23:51Sliceline Internals: Slice Errors
- 25:31Open Source Package
- 26:54Code Example
- 30:40Takeaways
related talks
30mReal-time bot mitigation with machine learning in Flink
01h 00mRock ‘Em Sock ‘Em Robots: Bot Swatting Like the Pros
15mRealtime bot detection with Flink
23mBot or human? Detecting malicious bots with machine...
49mA Real-World Example of Using Network Behaviour to Detect Cyber-Compromise
25mReal-Time Fraud Detection at Scale
16mAnalysing Real time Activity Streams for Security
59mCatching Malware En Masse: DNS and IP Style