Your AI Agent Passed Every Test. Your Humans Still Rejected It. Now What?
summary
When agents pass technical tests but users still reject them, the missing work is the human quality layer: measurable visibility, reproducible artifacts, and named decision rights. Teams instrument verification frequency, overrides, abandoned runs and log every “no” as a decision defect with an owner. Use the five operational questions — Can I see it? Can I understand it? Can I influence it? Can I own it? Can I trust it appropriately? — and run success, ambiguous and failure scenarios (example: incident‑response rollback) to make release criteria concrete.
AI-written summary · about
chapters (63)
- 0:00The Agent Passed Every Test and the Humans Still Rejected It
- 2:12Speaker Intros - Jill Hosmer Jolley, Faculty at California State University
- 4:15Starting With the Person Who Doesn't Trust the Agent Yet
- 4:45Skepticism as a Reasonable Request, Not Resistance
- 5:15What We Already Test - and What It Doesn't Tell Us
- 5:45Naming the Gap: The Human Quality Layer
- 6:16Five Plain Questions: See, Understand, Influence, Own, Trust
- 6:46What Rejection Actually Looks Like - and It's Quiet
- 7:17Running the Old Process and the New One Side by Side
- 7:47The Launch Report Looks Healthy While the Workflow Gets Slower
- 8:17What to Measure: Verification Frequency, Overrides, Abandoned Runs
- 8:48Accountability Without Control Is Psychologically Unsafe
- 9:19Two Lenses: Human Experience and Engineering Response
- 9:49Different Roles Need Different Proof
- 10:51Translating Human Needs Into Buildable, Testable States
- 12:23An Agent Never Arrives in an Empty Workflow
- 13:25The Human Contract for Every Consequential Step
- 13:55Recommendation vs Approval vs Execution vs Rollback
- 15:36The Readiness Data: Uneven Shared Understanding
- 17:13Treat Readiness as a Requirements Problem, Not a Comms Footnote
- 18:44Why Trust Comes Last on Purpose
- 19:47Mapping the Five Questions to Engineering Concerns
- 20:48Question One - Can I See It? The Incident Response Example
- 21:50A Concise Recommendation That Hides the Work
- 22:51Making Gaps Visible, Including a Failed Dependency Query
- 23:21Five Things a Human-Readable Run Record Must Show
- 24:22Visibility Should Respect Roles, Not Show Everyone Everything
- 24:52Test It With a Person, Not Only With a Schema
- 25:24Question Two - Can I Understand It? Explanation vs Verification
- 25:54Structuring an Explanation Around the Decision
- 26:58Reproducible Artifacts Over Claims of Invisible Reasoning
- 27:58When Two Sources Conflict, Surface the Disagreement
- 28:29Test Explanations When the Agent Is Wrong, Not Just Right
- 29:01Question Three - Can I Influence It? Disagreement as Design Input
- 29:31The Commander Who Agrees With the Fix but Not the Scope
- 30:33Influence Across the Lifecycle: Before, During, After Execution
- 31:33Undo vs Compensating Action - Don't Use the Same Words
- 32:04Making Control Real in the Backend
- 32:34Testing the Ugly Moments: Partial Success, Stale Auth, Cancellations
- 34:08A Stop Button That Only Hides the Progress Bar Isn't Control
- 34:40Question Four - Can I Own It? Authority vs Exposure
- 35:10Which Human, in Which Role, With What Authority?
- 35:40Encoding Accountability: Recommender, Approver, Executor, Escalation Owner
- 36:42Approval Authorises the Exact Action, and a Plan Change Invalidates It
- 37:13Ownership Includes Capacity and Psychological Safety
- 38:14Handoffs Across a Shift Change
- 39:19Question Five - Can I Trust It Appropriately?
- 39:51Autonomy Needs Evidence, Not a Successful First Incident
- 40:22Suggest, Collaborate, Act With Approval, Bounded Autonomy
- 40:53Downward Adjustment Is Part of the Collaboration
- 41:54Watching Both Over-Reliance and Under-Reliance
- 43:28Putting the Five Questions Together as One Layer
- 44:28Why the Conditions Depend on Each Other
- 44:59Running Success, Ambiguity and Failure Scenarios
- 46:02Turning Review Evidence Into Release Criteria
- 47:33What Happens When the AI Is Wrong - and Who Gets Blamed
- 49:06Psychological Safety Without Removing Accountability
- 49:36What to Do Next Week: One Workflow, Three Scenarios
- 51:10Walking the Five Questions With Evidence
- 52:13Logging Every "No" as a Decision Defect With an Owner
- 52:45Build the Agent, Build the Workflow, Build the Human Quality Layer
- 54:16Q&A: What Are Humans Rejecting - Trust, Tone or Workflow Fit?
- 57:53Q&A: Have You Ever Killed a Technically Perfect Agent?
related talks
43mTesting Agentic AI Applications: Beyond Traditional QA
20mThe Three Ways AI Agents Fail. Here's How to Fix Yours.
32mQuality engineering in the AI age: From testing AI to leveraging GenAI
29mBuilding Trust in AI Agents
31mIntroducing Agent Testing
37mAI agents testing How to evaluate the unpredictable
32mYour APIs Are Not Agent-Ready: How to Test What AI Agents Actually Do
33mEvaluating Your AI Agent