AI Architecture Thinking: From Idea to Impact
summary
A concrete agentic QA stack is built and shown working end-to-end: a feature planner (Opus frontier model) calls a smaller locator inspector (Playwright + MCP, Sonet/Haiku experiments) and a quality reviewer that gates commits; the system outputs BDD feature files, locators and step defs, runs deterministically outside the LLM, records traces in Codex/Slack hooks, and uses LambdaTest for execution. Architecture enforces governance, memory pruning, tool contracts and six autonomy levels; the demo succeeds on a happy path.
AI-written summary · about
chapters (41)
- 0:00Start
- 0:05Session Opens - AI Architecture Thinking: From Idea to Impact
- 0:36The Brilliant Demo That Never Reached Production
- 1:37Speaker Intro - Safeena Banu, Principal Technical Evangelist, Elsevier
- 3:45The Use Case: A Quality Engineering Agentic Stack
- 4:48Fixing an Architectural Mistake Early Is Cheap; Later It Isn't
- 5:50The Full Stack: Planner, Author, Deterministic Runner, Healer, Deploy Monitor
- 6:51Three Agents to Watch: Feature Planner, Locator Inspector, Quality Reviewer
- 7:51A Cheaper Model for Inspection, a Frontier Model for Planning
- 8:21The Deterministic Layer the Model Never Touches
- 9:24Live Demo: Running the Agentic Stack Inside Codex
- 10:57Requiring Approval Before Any URL Navigation
- 12:29The Quality Review Gate as the Final Check
- 14:34Second Demo: A Personal Monthly AI Learning Map
- 17:12An Agentic QA System Is Not a Model in a Wrapper
- 17:42Deciding What Should Be One Agent vs Sub-Agents
- 18:42Building the Cake Bottom-Up - Layer One: Governance and Observability
- 19:45Layer Two: Context and Memory
- 20:15Layer Three: Tools - the Hands of the LLM
- 20:46Layer Four: Reasoning and Model Routing
- 21:16Layer Five: Orchestration; Layer Six: The Interface
- 22:18Defining Guardrails by Access and Risk
- 22:48Autonomy Is Just Delegated Control Flow
- 24:19Micro Pipeline, Router Plus Workers, or Multi-Agent
- 25:19Multi-Agent Is a Cost, Not an Achievement
- 25:51Models for Judgement, Code for Truth
- 26:51Tools Are API Contracts - Define the Input and the Output
- 27:53A Tool Accepting Natural Language Authority Moves Your Security Boundary
- 28:54The Model Proposes a Patch - It Cannot Apply One
- 29:27Context Is a Finite Resource
- 31:30Memory Design Depends on Access Frequency
- 32:32Governance in Practice: Six Autonomy Levels for Healing
- 33:35The Real Question: Who Is Allowed to Believe the Heal?
- 34:36A Passing Test Is Not a Good Eval
- 35:37Instrument Before You Tune the Prompt
- 36:39Cost at Scale: A Cheaper Model in a Decision Step Is Not Cheaper
- 37:41A Security Warning: Don't Trust a Skill Because It Has Stars
- 40:13Test Artifacts Are Attacker-Influenced Inputs
- 41:13The Final Synthesis: Security, Observability, Traces, Evals, Cost, Kill Switch
- 41:44The Takeaway: Aim for the Least Autonomous System That Works
- 42:14If You Can't Name the Contract Owner, Failure Mode, and Success Metric, It's Not Ready
related talks
48mBeyond Bugs: How AI Turns QA into Full-Cycle Engineers
44mAspect-oriented Architecture
48mDown the Ivory Tower towards Agile Architecture
01h 34mThe Intersection of Architecture and Implementation
53mTest architecture is a real thing
31mModern Architecture for AI-Augmented Development Teams
58mTesting Software Architecture
51mTesting Software Architecture