Subutai Ahmad, VP of Research, Numenta
summary
Numenta publishes an open-source Numenta Anomaly Detection Benchmark (NAB) that streams ~350,000 points across 47 real-world datasets and 11 artificial streams, scoring detectors with a windowed, sigmoidal function that rewards early detection and penalizes false positives; it offers three application profiles to weight FP vs FN. NAB runs detectors online (no lookahead, no per-series tuning), is GPL on GitHub, and initial seed results show Hierarchical Temporal Memory scores ~64/100 versus chance 16.8 while Skyline and Twitter methods score lower.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:50REAL-TIME ANOMALY DETECTION
- 2:16EXAMPLE: PREVENTATIVE MAINTENANCE
- 3:11YET ANOTHER BENCHMARK?
- 4:05NUMENTA ANOMALY BENCHMARK (NAB)
- 5:06EXAMPLE: LOAD BALANCER HEALTH
- 5:40EXAMPLE: HOURLY SERVICE DEMAND
- 6:21EXAMPLE: PRODUCTION SERVER CPU
- 7:01HOW SHOULD WE SCORE ANOMALIES?
- 8:43WHERE IS THE ANOMALY?
- 9:17NAB DEFINES ANOMALY WINDOWS
- 9:29SCORING FUNCTION
- 10:34OTHER DETAILS
- 12:47TESTING ALGORITHMS WITH NAB
- 14:03NAB V1.0 RESULTS (58 FILES)
- 14:52DETECTION RESULTS: CPU USAGE ON PRODUCTION SERVER
- 15:48DETECTION RESULTS: MACHINE TEMPERATURE READINGS
- 16:21DETECTION RESULTS TEMPORAL CHANGES IN BEHAVIOR OFTEN PRECEDE A LARGER SHIFT
- 17:09SUMMARY
- 18:50NAB RESOURCES
related talks
43mDetecting Anomalies in Streaming Data
41mApplying Foundational Models for Time Series Anomaly Detection
52mDevil in the Haystack
29mAnomaly Detection
28mUnsupervised Real-Time Stream-Based Novelty Detection Technique
30mRealtime Time Series Anomaly Detection in Production
29mInterpretable Anomaly Detection for Numerical Data in Python Using PyNomaly
30mAnomaly Detection with Apache Pinot & ThirdEye