Automating UX Compliance Testing

on YouTube ↗31m

" summary

A Jenkins plugin is built to archive Selenium screenshots, compare them against a master image repo and report a pixel‑by‑pixel percent difference per endpoint; it runs on release candidates (not every commit), uses artifacts from CI, and applies thresholds so small diffs don’t fail the build. Comparison requires consistent resolution/OS, has caught actual UI regressions, and is slower than unit tests. Plans include tying comparisons into Selenium Grid Extras to compare image regions only and support video capture for debugging.

AI-written summary · about

" chapters (14)

1. 0:00Intro
2. 1:20Selenium timeline
3. 3:22Stack trace
4. 4:18Putting it together
5. 4:46How it works
6. 6:19Screenshot comparison plugin
7. 9:11Drawbacks
8. 11:27Image Archive
9. 12:29Jenkins Job
10. 13:33Screenshot Comparison
11. 14:35Live Demo
12. 18:40What doesnt compare
13. 19:22CI Independent
14. 20:12Selenium Extra Grid

" related talks

• 25mUsing Jenkins to Display Selenium Failures & UX ComplianceDavid Tolley · Jenkins User Conference San Francisco 2012
• 39mSpot the difference: automating visual regression testingViv Richards · SeleniumConf Chicago 2018
• 37mSpot the Difference; Automating Visual Regression TestingViv Richards · TestCon Vilnius 2018
• 12mIntegrating visual testing into the build pipelineShokoofeh Hesari · NDC Oslo 2018
• 37mThe Colorful World of Visual Test AutomationOnur Baskirt · Selenium Camp 2017
• 27mVisual UI Testing – Testing the Last MileCraig Cavalier · React Boston 2019
• 34mENABLING CONTINUOUS DEPLOYMENT OF PIXEL-PERFECT USER INTERFACESStefano Rago · Incontro DevOps Italia 2018
• 18mImplementing a Visual CSS Testing FrameworkJessica Dillon · CSSConf Oakland 2014