The Great Doubt: What Building an AI Agent Taught Us About Trust
summary
Built a production AI observability agent called Grafana Assistant that integrates Prometheus, Tempo and K6 and can raise PRs; evaluation uses an internal harness LLMspec with a golden dataset plus OpenTelemetry traces for per-conversation debugging. Introduces pass-at-three flaky-test criteria, finds an LLM judge that “cheats,” a 75% evaluator pass rate that needs context, and system prompts (≈20k tokens) that sometimes make models worse across Opus 4.6→4.7. Online evals and OllyBench leaderboard (Opus top, Qwen 3.6 ≈#14) help but multi-agent handoffs and feedback coverage remain unsolved.
AI-written summary · about
chapters (16)
- 0:00AI agent and trust challenges
- 2:06Grafana Assistant AI agent
- 5:16Critical agent failure in production
- 6:45Great Doubt from Japanese philosophy
- 9:15LLMspec internal benchmarking
- 10:21Flaws in tests and reliability
- 13:54Agent Observability for monitoring
- 15:39Doubt in evaluation scores
- 17:48Prompt analysis and model issues
- 20:59Value of system prompts
- 22:43Serena incident testing
- 23:47Self-improving AI loop oversight
- 25:11OllyBench observability benchmark
- 26:57Language limitations and gaps
- 28:00Trust through experimentation
- 29:24Conclusion and path to awakening
related talks
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
47mBuild Trustworthy AI Agents powered by Evals
33mEvaluating Your AI Agent
01h 20mMind the Gap (In your Agent Observability)
33mWhat Did My Agent Do? Observability and Accountability for AI Agents
25mWho Let the Bots Out? A Guide to Evaluating AI Agents
10mLightning Talk: Building AI That Ops Teams Actually Trust
35mThe Agentic AI Engineer