Challenges and Pitfalls of Applying GenAI in Monitoring
summary
Experiments show LLMs can parse messy logs into OpenSearch bulk JSON and do root‑cause reasoning but not at scale: Claude and an Olama Quen model extract timestamps/hosts/messages and produce ingestible JSON on a MacBook M4, slower than regex/grok; root‑cause output correctly finds slow DB queries, connection‑pool exhaustion and suggests an index on email. MCP connects LLMs to OpenSearch for natural‑language queries. Time‑series model TimeGPT misses 13 injected anomalies and reports 22 false positives, precision effectively zero.
AI-written summary · about
chapters (9)
- 0:00Introduction to GenAI in monitoring
- 2:17What changed in GenAI since 2023
- 6:10GenAI use cases in monitoring
- 8:16Demo: Parsing unstructured log data
- 12:32Demo: Root cause analysis with logs
- 18:57Demo: Data exploration with MCP
- 28:50Demo: Time series anomaly detection
- 32:45Conclusion and future outlook
- 33:47Q&A session
related talks
41mGenerative AI: bliss or miss?
44mAI Foundations: Mitigate Risks and Boost SOC Efficiency
32mMonitoring in the era of Generative AI, LLVMs, and embeddings
20mGenAIOps - OpenSearch for AI & GenAI Platform Observability
25mThe (mis)use of GenAI in Data Analytics
24mHow OpenTelemetry Helps Generative AI
01h 04mWhat are we thinking in the age of AI?
29mKeynote: LLMs, what are they good for?