MLGuard -- Detecting Malicious Web Requests using a Serverless-based Machine Learning System
Talkat Cloud Native Security Day 2019 · on Monday, 2019-11-18· at 11:55· Marriott Marquis San Diego Marina - San Diego Ballroom A, 333 W Harbor Dr·
summary
MLGuard is a serverless pipeline that flags malicious IPs by training an Isolation Forest on per-IP frequency distributions of HTTP response codes derived from ELB logs. ELB logs land in S3, an enrichment Lambda adds geo and security context and writes to Elasticsearch, a daily containerized job in ECR trains on millions of requests and deploys via Amazon SageMaker, and a periodic Lambda hits the SageMaker endpoint to score IPs, post Slack alerts and fire webhooks to add offenders to WAF. Deployment blocks thousands of bad IPs (339 blocked in one month).
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:20Frame-accurate communication
- 2:03Central asset organization & sharing
- 2:35Secure sharing you can customize
- 4:49Frame.io Platform with WAF
- 5:44WAFs are Easy-to-Evade
- 6:39Anomaly Detection
- 7:51Introducing MLGuard
- 9:23Serverless: Function-as-a-Service
- 11:13Benefits of Serverless
- 12:50Serverless Design Pattern
- 13:00Event-driven Serverless Functions
- 14:07Periodic-Invocation Serverless Functions
- 15:52MLGuard: Collect Logs
- 17:55MLGuard: Algorithm Overview
- 20:32MLGuard: Peak into Features
- 23:10MLGuard: Prediction
- 24:16MLGuard: Notification to Analyst
- 25:42Metrics - Last Month
- 27:07Take Away
related talks
39mDetecting Website Intrusion and Account Compromise with Machine Learning
21mMachine Learning and Cyber Security - Detecting malicious URLs in the haystack
41mWill Machine Learning Replace The WAF?
33mServerless Security: New Risks Require New Approaches
40mServerless Architecture Security Patterns for Securing the Unseen
29mAutomating Threat Detection and Remediation
49mProtecting the Web at a scale using consul and Elk
50mAutomated Serverless Security Testing: Delivering secure apps continuously