Transforming Observability: Inside New Relic's AI Assistant
summary
New Relic Grok uses a large language model to translate natural language into nrql query by first choosing relevant tables, pulling schemas, retrieving 15 similar examples from a vector database, and assembling a prompt; that pipeline calls the LLM about four times per request. It executes queries (example result: 997 failed transactions) and checks anomaly APIs, renders charts, and returns text. Measured latencies show ~4s to intermediate token, ~11s to first token, ~31s to final token, and a cost model of about $4,900/month for 100 users at 10 requests/day.
AI-written summary · about
chapters (12)
- 0:00New Relic and Gro intro
- 3:37Gro’s UI demo
- 5:03Gro’s four core capabilities
- 7:55Gro tool selection logic
- 9:01Natural language to NRQL
- 11:09Answering questions from documentation
- 12:57Building domain AI challenges
- 17:18User question feedback
- 18:22Prompt refinement edge cases
- 21:10Gro latency bottlenecks
- 22:36LLM cost management
- 23:19Gro future plans
related talks
07mUnderstanding AI performance and cost management with New Relic
08mThe Future of Observability: Trends, AI, and New Relic’s Vision f...
38mAI Monitoring Mastery: Optimize Performance, Ensure Compliance, and Cut Costs
19mDo We Still Need to “Observe”? the Future of AI & Observability
48mHands-on guide to monitor your API-driven AI/LLM applications
28mDO WE STILL NEED TO "OBSERVE" THE FUTURE OF AI & OBSERVABILITY
37mChallenges and Pitfalls of Applying GenAI in Monitoring
33mWhere Are We Going With Observability?