Lightning Talk: Summarizing the Noise: LLM Observability With Open Data Hub, VLLM...
summary
An open-source AI observability summarizer built on Open Data Hub deploys LLMs via vLLM and KServe, scrapes metrics from Prometheus/Thanos, traces from Tempo, and logs from Loki, then uses Correlator to link signals and generate PromQL and human-readable reports (PDF/HTML/markdown). It claims to cut a 30‑minute manual diagnosis into seconds, handles the noise of 3,000–4,000 Kubernetes metrics plus 50+ vLLM and 30+ GPU metrics, and shows a working UI that produces actionable root‑cause summaries.
AI-written summary · about
chapters (10)
- 0:00AI observability challenges
- 1:06Metrics aren't enough for LLM monitoring
- 3:10Introducing AI observability summarizer
- 4:14Key features of AI observability tool
- 5:16Chat with Prometheus in practice
- 5:37Architecture of AI observability system
- 6:41Live demo: AI summarizer
- 8:46Analyzing VLLM with AI insights
- 9:28Interpreting latency with natural language
- 10:11Key takeaways and next steps
related talks
07mLightning Talk: Demystifying Kubernetes Observability with Generative AI and LLMs
38mThe Art of Compression: Crafting Insightful Summaries with LLMs
13mLightning Talk: Observability-First DevSecOps: Building Resilient Multi-Cloud P...
23mObservability for LLMs: Understanding What’s Happening Under the Hood
10mLightning Talk: LLM-Driven Applications: Intro to LLM Agents in Kubernetes
23mFrom Hallucinations To Hardware: Diagnosing LLM Failures
05mLightning Talk: Machine Learning Observability with Prometheus
13mLightning Talk: A Drop-in System To Accelerate Metrics Observability by 100x Us...