Building a screenshot testing pipeline that scales
summary
Pipeline uses Paparazzi to generate screenshots and reg-suit to diff them against a baseline stored in a GCS bucket, producing a web report and a PR comment. Design system adoption yields ~700 screenshots (two per test for light/dark), CI pre-merge runs ~8 minutes and screenshot generation is ~30s. Tradeoffs: Paparazzi is fast and reliable but misses some Compose behaviors (dynamic text resizing) and had a build-cache bug fixed in v1.3; external storage avoids git merge conflicts for large-scale visual changes.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:46Manual UI testing challenges
- 7:15Screenshot testing goals
- 12:51Screenshot testing frameworks
- 18:39Storing screenshots and diffing
- 20:01CI/CD pipeline
- 25:21Writing screenshot tests
- 29:18Adoption and results at Mercari
- 33:13Limitations and future work
related talks
29mScale Up Your Snapshot Testing Without the Friction
23mAll your Compose @Previews to screenshot tests without instrumentation
35mEasy screenshot testing with Compose
58mUnit Testing with Screenshots: Breaking the Sound Barrier
18mImplementing a Visual CSS Testing Framework
29mHow to set up a pipeline with visual unit tests for your app
37mSnapshot testing (and more!) with Paparazzi
34mENABLING CONTINUOUS DEPLOYMENT OF PIXEL-PERFECT USER INTERFACES