Why observability requires the marriage of AI, metrics, and logs
summary
Argues that observability succeeds only by combining metrics, logs and machine learning into a single platform that runs AI at the edge to convert high-volume telemetry (thousands of events/sec and up to ~100,000 metrics/sec) into actionable incidents. Describes information‑theory filtering where 1% of data -> 80% of incidents, vertex‑entropy and convolutional similarity for clustering, a requirement of ~2,000 datapoints to learn a metric pattern, and deployed pieces that surface root cause, feed supervised models, and drive closed‑loop automation (some customers remove their entire NOC).
AI-written summary · about
chapters (14)
- 0:00Introduction and personal journey
- 1:25AI in service assurance
- 3:11Evolution of digital infrastructure
- 6:49Digital transformation impact
- 10:04Complexity and service expectations
- 13:37Limitations of rules-based monitoring
- 16:51AI as a solution for monitoring
- 21:17Metrics and logs in observability
- 26:24AI-driven observability architecture
- 32:11AI Ops manifesto for observability
- 37:16MOOC soft product and AI Ops
- 41:18AI algorithms and data processing
- 48:35Unique value and customer success
- 49:22Q&A session
related talks
42mMachine Learning In Observability Systems: From Insights To Action
18mAI Observability: How To Fix Issues With Your ML Model
13mObservability Where are We Headed?
36mObservability? It's a Data Analytics Problem, You Fool!
35mBeyond Logs, Metrics and Traces. The Data Analytics Paradigm Shift in Observability — Dotan Horovits
29mObservability; a gentle introduction
27mAlert Fatigue to Action: Observability as the Backbone of Practical AIOps with OpenSearch
30mObservability is More than Logs, Metrics & Traces