Why Are Your AI’s Decisions Hard To Explain: Trace Every Decis...
summary
Argues that observability must trace every agent decision, proposing one span per decision recorded via OpenTelemetry and ingested into OpenSearch + Data Prepper; captures reasoning as span events, tool calls, attributes (model, tokens, confidence) and treats token spend as quality metric. Demonstrates a working pipeline using Amazon Bedrock, Bedrock Agent Core and Strands, surfaces a run with 36 spans, ~49s end-to-end latency, token counts and per-span metadata, and shows queryable dashboards and alerts for failures, hallucinations and cost spikes.
AI-written summary · about
chapters (7)
- 0:00AI agent observability
- 1:04Explainability gap in decisions
- 5:23Decision tracing solution
- 9:57OpenTelemetry for decisions
- 15:15AI agent with AWS tools
- 23:15Demo of decision tracing
- 25:51Q&A
related talks
33mWhat Did My Agent Do? Observability and Accountability for AI Agents
31mDeep Dive To AI Agent Observability
22mYour AI Agent Has No SLO
30mTutorial: Building an Observable AI Agent With an Open Source Stack
27mObservability as a foundation for AI-enabled platforms
40mOptimizing AI Applications for Production with Observability
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry
17mContext Graphs for Explainable, Decision-Aware AI Agents