Architects of Trust: Redefining Quality Leadership in an Agentic World
summary
Classical coverage metrics (94.2% test coverage) hide a midnight recursive prompt loop that costs $400,000 before alerts fire, so trust becomes a measurable engineering metric. The session presents an Empirical Trust Framework that maps statistical risk boundaries to SLAs, uses statistical evaluation and deep evals across agency levels (assistant, coordinator, autonomous), and advocates Chaos Audits, guardrails, feature flags and quick rollback. Results: reduced production exceptions by ~98% and engineer burnout by ~65% in running pilots.
AI-written summary · about
chapters (10)
- 0:00What Should Quality Leaders Measure in the Age of Agentic AI?
- 2:43Why High Test Pass Rates Can Still Hide AI Failures
- 4:47Why Traditional Testing Fails for Probabilistic AI Systems
- 7:58How Do You Measure Trust in Agentic AI?
- 11:03Which Metrics Reveal Whether an AI Agent Is Truly Reliable?
- 14:08How Should Teams Measure Risk as AI Agents Gain Autonomy?
- 18:15How Must Quality Leadership Change for Agentic AI?
- 23:23How Chaos Testing and Statistical Evaluation Build AI Trust
- 27:01What Skills Will Define the Future Quality Engineer?
- 34:12How Does a Quality Engineer Become a Trust Architect?
related talks
32mDo You Trust Me? A look at Trust, Time, and Teams
13mTrust and Teams
21mBuilding the Trust Layer for the Agentic Era
20mRefactoring Trust on Your Team
25mLLM Agents for Site Reliability
16mTrust by Design: Building Reliable AI for Sensitive Environments
12mRebuilding trust & improving understanding between business & engineering
20mBuilding a Culture of Reliability