Real World Lessons from Scaling Test Automation
summary
Describes running Playwright at enterprise scale across a monorepo with ~14,000 tests, pairing Playwright MCP and CI (Azure DevOps/GitHub) with planner/generator/healer agents that generate tests from requirements, classify failures, and propose repairs. Reports pilot numbers: 60% faster PR feedback, 40% drop in manual test repairs, 90% accuracy classifying bug vs test failures. Covers risk‑based test selection using dependency/AST analysis, self‑healing via DOM/snapshot comparisons, and strict governance with human approval and least‑privilege sandboxes.
AI-written summary · about
chapters (13)
- 0:00Scaling Test Automation Across Large Engineering Teams
- 4:19Where Test Automation Breaks as Teams Scale
- 7:24From Scripted Tests to AI-Driven, Self-Healing Automation
- 9:56Scaling Playwright Across 14,000 Tests in a Monorepo
- 14:31Combining AI Reasoning With Deterministic Test Execution
- 18:04Building an Autonomous AI Test Automation Workflow
- 23:10Improving Test Coverage From Business Requirements
- 26:15Risk-Based Test Selection With Code and Dependency Analysis
- 29:57How Self-Healing Tests Diagnose and Repair Failures
- 31:31AI-Powered Root Cause Analysis for Test Failures
- 33:37Governance and Safety for AI Test Automation
- 37:12Results and Lessons From Scaling AI Test Automation
- 44:24A Phased Roadmap for Enterprise Test Automation
related talks
18mAt the top of the pyramid: Playwright testing at scale
48mSupercharged Testing: AI-Powered Workflows with Playwright + MCP
21mAt the top of the pyramid: Playwright testing at scale
12mPlay It Right With Playwright
29mPlaying your tests wright
35mState of end-to-end testing for modern web applications
43mReal World Playwright Case Studies That Will Change Your Perspective!
28mFrom generic to specific test automation architecture with Playwright