~/panel

Panel Discussion: Agentic Engineering and Quality in Healthcare

on YouTube 47m
Panelat TestMu Conference 2026 · 2026-08-19..08-21· [ai-assisted-development]

speakers

name:Chad Ullman
links:none on file
name:Rohit Jaiswal
links:none on file
name:Mark Kroh
links:none on file

resources

video:https://youtu.be/2JHvJ7dktko
slides:none on file

summary

Panelists report concrete results: a $250K consultancy scope is replaced by augmented teams, test authoring in the IDE and first‑pass code review save hours, and iterative self‑improvement of LLM‑written suites mutation score plateaued at 53% until given concrete failures — show it the breaks and it jumps to 90%. Validation shifts to protocols and continuous auditing, since pinning the model is a dependency with a clock; operational fixes favor reversibility and one flag per flow with a removal date.

AI-written summary · about

chapters (52)

  1. 0:00Agentic Engineering and Quality in Healthcare
  2. 0:35Panel Intros - Chad Ullman, Rohit Jaiswal, and Mark Kroh
  3. 3:39Beyond the Hype: Where Agentic AI Actually Moves the Needle
  4. 4:09Conservative by Nature - and Why That Wasn't Doing Nothing
  5. 4:41Replacing a $250K Consultancy Engagement With Augmented Teams
  6. 5:11AI as a Superpower, Not a Super Replacement
  7. 5:41Test Authoring in the IDE and Automated First-Pass Code Review
  8. 6:12The Mutation Score Experiment: Self-Improvement Plateaus at 53
  9. 6:43Show It the Specific Breaks and the Score Jumps to 90
  10. 7:13LLMs Aren't Short on Capability - They're Short on Feedback
  11. 7:44Amplifying the Tester and Tracing Coverage Back to Requirements
  12. 9:47Point-in-Time Validation Doesn't Work for Systems That Adapt
  13. 10:18Sign Off on the Protocol, Not the Artifact
  14. 10:48Borrowing the FDA's Predetermined Change Control Plan
  15. 11:19Your Eval Suite Becomes Your Regression Suite
  16. 11:49Pinning the Model Is a Dependency With a Clock on It
  17. 12:20From One-Time Sign-Off to Continuous Auditing
  18. 13:51Traceability When Something Fails Is the Part That Stays
  19. 14:53Understanding the Contract: Requirements Become More Important, Not Less
  20. 15:55What "Safe to Ship" Means in a Highly Regulated Domain
  21. 16:59The Duplicate Claim That Used to Take Months to Surface
  22. 18:01Testing Distributed Across the Pipeline - Everyone Owns Quality
  23. 19:32The Engineering Director's Answer: How Fast Can I Un-Ship?
  24. 20:34Investing in Reversibility: One Flag Per Flow With a Removal Date
  25. 22:06Synthetic Data vs Real World Edge Cases
  26. 22:37De-Identified Production Data and Querying It With AI
  27. 23:38Data Mining Instead of Synthetic Generation
  28. 24:08The Structural Problem: Privacy Deletes the Tail of the Distribution
  29. 24:38Stop Measuring Fidelity - Measure the Synthetic-to-Real Delta
  30. 25:08"Vintage": Every Synthetic Patient Is Born Today
  31. 25:38You Can't Test Your Way to Edge Case Coverage
  32. 26:10Feeding Production Incident Tickets Back to Generate Edge Cases
  33. 27:40Why De-Identified Data Goes Stale and Breaks Your Automation
  34. 28:43Human in the Loop vs Human on the Loop
  35. 29:13Why Human Review at Agent Scale Degrades Into Rubber Stamping
  36. 29:43The AI Reviewer That Silently Stopped Reviewing
  37. 30:15Attribution and Giving the Reviewer a Signal, Not a Pile
  38. 30:45Three Deliberate Places for Humans: Before, During, After
  39. 31:16Never Bypass the Loop - Building Fundamentals in New Testers
  40. 33:18The Future of the Automation Engineer
  41. 34:18Becoming a Communicator Instead of a Coder
  42. 35:18Four Priorities: Oracle Design, Eval Design, Reading a Suite Critically
  43. 36:19The Most Underrated: Test Data Generation as Engineering
  44. 36:51Business Domain Understanding as the Real Moat
  45. 37:22One Strategic Objective to Take Back Tomorrow
  46. 38:25Invest in Your People as Much as Your Technology
  47. 38:56Adopt the Agents but Own the Context
  48. 39:27Know Which of Your Controls Are Mechanisms and Which Are Just Sentences
  49. 39:57The Replit Incident: A Code Freeze That Was Text, Not a Permission
  50. 40:28Go Verify Every Required Check Actually Runs, Not Just Passes
  51. 42:01Q&A: HIPAA Compliance and Agent Workflows
  52. 44:05Q&A: Are We Creating an Accountability Gap?

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.