This Lying Has To Stop: Keeping AI Honest with OpenTelemetry
summary
An automated agent called Commit Story runs on git commit, harvesting code diffs, commit messages and every assistant chat so AI-generated journal entries appear automatically. Using Cloud Code plus OpenTelemetry sent to a Datadog backend, traces and correlated logs reveal that 448 raw chat messages (including huge tool calls) reach generators instead of the intended filtered 174 because a recent grouping by chat ID bypasses the filter. A small code change restores filtering and traces confirm the fix.
AI-written summary · about
chapters (9)
- 0:00AI honesty with OpenTelemetry
- 1:04Commit Story journaling demo
- 7:21What is OpenTelemetry
- 9:26Development telemetry benefits
- 15:45Instrumenting functions with OpenTelemetry
- 18:11Telemetry improves debugging
- 20:41Telemetry drawbacks and unknowns
- 22:07Future of automated instrumentation
- 22:28Conclusion & Q&A
related talks
20mTracking AI generated code with Git
33mWhat Did My Agent Do? Observability and Accountability for AI Agents
29mLeadership in AI-assisted engineering
28mThe Hard Truth About AI Agents
31mDeep Dive To AI Agent Observability
50mPanel: Beyond The Vibe: Code Quality First
01h 02mObserving AI Applications with OpenLit and OpenTelemetry
19mYour Biggest User Isn't Human: Designing DevTools for AI Agents