Developing new findings using machine learning in Amazon GuardDuty
summary
Amazon built contextual encoder‑decoder models (autoencoders) to model "normal" behavior across CloudTrail, S3 data events and RDS login logs, then scores tens of billions of events per minute and flags anomalies for post‑processing, aggregation and enrichment; teams inject synthetic attacks and run months of manual validation to tune thresholds. Results: a sharp reduction in benign noise and more high‑value alerts, currently shipping as 13 GuardDuty findings covering hundreds of APIs and services with eBPF‑based EKS runtime visibility.
AI-written summary · about
chapters (8)
- 0:00Introduction to GuardDuty
- 3:29Data visibility and sources
- 5:36The challenges of anomaly detection
- 12:39Machine learning methodology
- 22:19Production and post-processing
- 25:05Model tuning and evaluation
- 30:20Improving actionable insights
- 46:12Key takeaways and conclusion
related talks
31mBuild threat detection models using ML
15mIP Detection at Scale: CI/CD Meets Cloud-Native Enforcement
28mDetect abnormal equipment behavior by analyzing sensor data
01h 06mAmazon Machine Learning to Predict Responses
26mServerless Architecture for Product Defect Detection
29mAI-Powered Anomaly Detection in Cloud Data Pipelines
44mBest practices for analytics and generative AI on AWS(ANT302)
38mAnomaly detection in the cloud: how to detect issues in data [...]