Challenges in UI Automation and How to Solve Them
summary
Claims that web UI testing is solved with selector-based methods but breaks on Shadow DOM, iframes, canvas and focus quirks; demos show Cypress running fast in CI yet failing on a readOnly/focus case, and Taiko handling relational selectors like “text box below submit.” Visual approaches use OCR/opencv and whole-screen clicks to handle canvas and native apps; tools mentioned: Percy, eggplant, Appium, Playwright. Caveat: self-healing tests reduce flakes but can mask real structural bugs and add maintenance.
AI-written summary · about
chapters (14)
- 0:00UI automation challenges
- 1:38UI beyond screens
- 5:05Visual vs code selectors
- 9:26Legacy UI quirks
- 13:20Cypress selector demo
- 18:45Selector-based tools overview
- 19:47Flutter cross-platform
- 21:51Smart selectors for tests
- 25:52Taiko relational demo
- 28:42Visual regression testing
- 31:14AI visual testing
- 34:43Visual tools landscape
- 36:07AI/ML future of UI automation
- 37:56Q&A
related talks
21mYour Tests Lack Vision: Adding Eyes to Your Automation Framework
56mYour Tests Lack Vision: Adding Eyes to Your Automation Framework
26mEffective DOM selector patterns. Interact like a user
27mVisual UI Testing – Testing the Last Mile
42mFunctional and Visual Testing for Web Applications
53mModern Web Testing: Going Beyond Selenium
18mModern Solutions for e2e testing
43mYour Tests Lack Vision: Adding Eyes to Your Automation Framework