NetBouncer: Active Device and Link Failure Localization in Data Center Networks
summary
NetBouncer actively probes paths using IP-in-IP and a packet bouncing strategy from servers to measure per-path loss, sends thousands of probes every five minutes, and runs a central controller plus an inference engine with a domain-aware regularizer. Deployed in Microsoft Azure for three years, it pinpoints link and device failures (it found a spine linecard bug causing ~15% loss), processes ~130 GB traces and finishes each 5‑minute analysis within 60s, and reports zero false positives and only a few false negatives in practice.
AI-written summary · about
chapters (13)
- 0:00Intro
- 2:18This is a true story
- 5:50Active probing system requires explicit and efficient probing
- 7:22Observation vs. inference from path probing to failures
- 8:50Real-world constraints complicate path selection
- 12:05Device failure detection
- 12:26Link failure inference: an optimization problem
- 13:00Real world data inconsistency induces false positives
- 14:36Evaluation questions
- 15:11Real cases spine router gray failure
- 16:10Accuracy comparison with previous algorithms
- 17:06NetBouncer algorithm performance
- 19:41NetBouncer has negligible averheads on the server side
related talks
32mActive Fault Finding in Networks
29mNetwork Fault Finding System: Packet Loss Triangulation
18mNetKeeper: Enhancing Network Resilience with Autonomous Network Configuration...
18mBayWatch: Practical Internet-Scale Topology Monitoring with Dynamic Bayesian Estimation
23mAutomatic Traffic Scheduling for Internet Connectivity Failures
29mFinding the Needle in a Haystack: Tenant-Level Network...
21mDiagnosing Application-network Anomalies for Millions of IPs in Production Clouds
20mNetwork Monitor: A Tale of ACKnowledging an Observability Gap