Monitoring Agentic Applications
Talkat Open Source Summit Japan + AI_dev: Open Source GenAI & ML Summit + Automotive Linux Summit 2025 · 2025-12-08..12-10·
summary
Project Monocle generates OpenTelemetry‑compatible traces for agentic apps with little to no code change, instrumenting AWS Strands and other SDKs to capture prompts, tool calls, token counts and agent delegations. A working demo shows an NBA agent implemented in about 200 lines of code where nondeterministic LLM behavior is diagnosed, fixed and regression‑tested via a pytest extension. The Monocle SDK sends JSON traces to storage or analysis tooling and a VS Code extension visualizes traces side‑by‑side with code.
AI-written summary · about
chapters (13)
- 0:00Agentic app monitoring intro
- 3:21Agentic design pattern
- 8:15Multi-agent architectures trade-offs
- 12:13Agentic frameworks and SDKs
- 15:59Debugging agentic apps challenges
- 18:02Monle tracing project intro
- 23:05Monle OpenTelemetry compatibility
- 25:56VS extension for Monle traces
- 27:57Debugging with Monle examples
- 30:13Testing agentic apps with Monle
- 33:43Monitoring agentic apps in prod
- 35:50Monle traces in Okahoo analytics
- 38:38Call to action community
related talks
22mMonitoring GenAI Applications
23mMonitoring GenAI Applications
33mWhat Did My Agent Do? Observability and Accountability for AI Agents
31mDeep Dive To AI Agent Observability
27mKeynote: Tracing the Agent's Mind: Extending OpenTelemetry for Deep...
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry
38mPractical introduction to OpenTelemetry tracing
47mUsing metric and traces to diagnose our cloud-native applications