Building Trustworthy AI: Open Source Monitoring in Action
summary
A working open source monitoring lab is presented that strings together Evidently, WhyLogs, MLflow, Truera plus metrics collectors, OpenSearch and Grafana dashboards to track data drift, output quality, toxic outputs and model lineage. A stress demo generates about 109 per minute and shows performance, cost and error-rate panels; pipeline alerts on drift and reconstruction logs for audits to meet upcoming EU AI Act requirements. The demo runs but shows stability and ROI questions to resolve before production.
AI-written summary · about
chapters (9)
- 0:00Introduction and speaker background
- 2:07Why trustworthy AI matters
- 10:15Risks of unmonitored AI systems
- 13:21Why monitor AI continuously
- 14:02Open source monitoring stack overview
- 18:18Demo focus on performance and cost
- 21:42Correlating metrics and logs
- 23:22Conclusion and key takeaways
- 26:52Q&A on ROI and business impact
related talks
40mOptimizing AI Applications for Production with Observability
38mAI Monitoring Mastery: Optimize Performance, Ensure Compliance, and Cut Costs
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
32mMonitoring in the era of Generative AI, LLVMs, and embeddings
30mFrom Black Box To Insight: Observability for AI Agents in Production
46mCan you trust your AI?
08mCan Monitoring Tools Pass the Turing Test?
40mKeep AI on Track: Guardrails + OpenTelemetry Observability