Diagnosing Application-network Anomalies for Millions of IPs in Production Clouds
summary
AND is a deployed Alibaba Cloud system that diagnoses application-network anomalies for millions of IPs by using TCP retransmissions as the single signal, collecting with eBPF-based kernel filters that drop over 90% of irrelevant events and keep per-microservice counters, clustering millions of time series into frequency/stability tiers, running lightweight real-time feature extraction and a GBDT routing model to assign blame, achieves minute-level detection at low CPU/memory overhead (single-core overhead below a few percent) and has operated in production for over three years enabling fast recovery.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:36Problem definition and goals
- 5:50Design challenges and insights
- 8:00System architecture and detection
- 10:37Optimizing eBPF filters
- 12:41Anomaly detection for millions of IPs
- 14:07Deployment results and CPU overhead
- 15:52Detection accuracy and recall
- 17:46Real-world case studies
- 19:16Conclusion and future work
related talks
08mKeynote: The Challenge and Opportunity to Build a Transparent Network
20mNetwork Monitor: A Tale of ACKnowledging an Observability Gap
23mWin the 0-Day Racing Game Against Botnet in Public Cloud
32mActive Fault Finding in Networks
19mNetwork X-Ray Vision: eBPF for Cloud-Native Observability
23mNetBouncer: Active Device and Link Failure Localization in Data Center Networks
21mHigh Network Reliability & Availability in FE & BE for Scalable Training Solutions
21mAAsclepius: Monitoring, Diagnosing, and Detouring at the Internet Peering Edge