The reality of testing in an artificial world
summary
An ad recommendation system uses machine learning to choose which creative to show via an ad recommendation API, trained by feedback from customers. Testing uses automated scripts that simulate 2,000 and 10,000 simulated users with randomized genders and a 65% acceptance rate for target ads. Models sometimes flip between recommending dresses or pants because the learner chases raw popularity not demographic signal, so the implementation fails certain business scenarios and requires larger-scale, repeatable automated tests and plain‑sense acceptance criteria.
AI-written summary · about
chapters (5)
- 0:00Start
- 7:00Learn How the System Learns
- 9:36Train the System
- 23:17Test Approaches
- 23:39Need for Automation
related talks
32mQuality engineering in the AI age: From testing AI to leveraging GenAI
01h 04mImpact of AI on #SoftwareTesting: Are Testers Ready? 🔮
01h 04mWhat Testers Need from Machine Learning and Artificial Intelligence
44mHow AI is Exposing the Existential Crisis of Software Testing
55mSurviving the AI Testing Apocalypse
28mOptimizing your A/B testing roadmap
09mTesting Strategies for Generative AI Applications
32mGjore Zaharchev: AI in Software Testing Expectation vs Reality