Automated Reduction of Flaky Tests

on YouTube ↗43m

" summary

Describes an automated layer that reduces flaky-test noise by enforcing test-writing rules with SonarQube (custom rules) run locally and in GitLab CI, plus a small service health-check framework that reads a JSON list of endpoints and expects HTTP 200 before tests run. The implementation analyzed 1,725 scripts in ~3.5 minutes, which removed most service-related failures and added immediate CI diagnostics; remaining flaky causes are mainly test data problems that are harder to automate.

AI-written summary · about

" chapters (7)

1. 0:00Automated test experience
2. 4:42Flaky test symptoms and causes
3. 10:09Prevention for services
4. 17:01SonarQube static analysis
5. 25:17Automation benefits and conclusions
6. 30:00Q&A
7. 40:24Closing

" related talks

• 46mThe state of the Art in Tackling Flaky TestsKohsuke Kawaguchi · DevDays Europe 2022
• 28mThe State of Art in Tackling Flaky TestsKohsuke Kawaguchi · cdCon 2022
• 26mKeeping Your Tests LeanMeaghan Lewis · SeleniumConf Berlin 2017
• 01h 15mTaking Control Of Your Test EnvironmentIoana Serban · CAST 2015
• 28mTest flakiness; methods for dealing with flaky testsJason Palmer · Assert(js) 2019
• 41m0Flake - Reaching reliable non-flaky testsItai Freindinger · DevOpsDays Tel Aviv 2018
• 34mZap the Flakes! Leveraging AI to Combat Flaky Tests with CANNIERDaniel Hiller · DevConf.CZ 2025
• 48mFlaky Tests 2.0Andrei Solntsev · DelEx Conf 2019