Panel Discussion: AI in Mobile QA - What Actually Works Today?
summary
Panel reports that nunu.ai runs vision-first agents that stream real devices, tap and swipe like humans, and currently lead public phone-use benchmarks; agents scale to thousands of devices in parallel, replace brittle scripted tests by adapting to UI changes, and produce full action+reasoning recordings for every run. Practical outcomes: verification vs exploratory discovery modes, persona-driven play (age personas), faster creation cycles, lower maintenance, and clear limits on nuanced fun or emotional judgment.
AI-written summary · about
chapters (31)
- 0:00What Actually Works in AI for Mobile QA Today
- 1:14Panel Intro - Jan Schnyder, Nicolas Muntwyler, and Justin Yuan
- 2:14Why Start With AI Agents for Mobile Games?
- 3:17From DeepMind-Inspired Research to Playing AAA Games on YouTube
- 4:49Why Automating Game Testing Was Historically So Hard
- 5:52A Fully Vision-Based Agent: No Accessibility IDs, No DOM
- 6:56Where Agents Match Humans - and Where Competitive Play Still Wins
- 7:56AI Doesn't Get Bored: The Real Advantage of Repeatable Runs
- 8:59Parallelism: Scaling to Thousands of Devices Instead of Testers
- 10:32Bugs vs "Is This Fun?" - The Part AI Still Can't Judge
- 11:04Games Are Made for Humans, So Keep a Human in the Loop
- 11:36Persona-Driven Play: Testing as a 70-Year-Old or a 10-Year-Old
- 13:09How Close Is AI to Nuanced Emotional Judgement?
- 16:43What Evidence Customers Want Back From an Agent Run
- 17:47Full Recordings, Every Action and Every Piece of Reasoning
- 18:18Verification Testing vs Exploratory Discovery Testing
- 19:51The Investor View: Why QA Was the Obvious First AI Beachhead
- 21:23Beyond Bugs: Monetization Verification, Ad Auditing, Age Verification
- 22:57What Agents Unlock That Traditional Automation Can't
- 23:58Maintenance: Why Agents Adapt Where Scripts Break
- 24:59Unexpected Use Cases: Ordering Uber Eats and Competitor Analysis
- 27:33Compliance Testing as an Emerging Mobile Use Case
- 30:06Accessibility Testing Moving From Last Check to First
- 31:36Why a Black-Box, Vision-First Approach Beats Injecting Into the App
- 33:41Feeding Context: Design Docs, Knowledge Base, MCP Integration
- 35:44How the Agent Avoids Random Clicking and Picks Real Flows
- 36:45Handling New Requirements Without Regenerating Every Test
- 38:50Creation Got Faster - Testing Became the Bottleneck
- 40:21Coverage: How Agents Decide the Execution Path
- 41:52How the QA Role Is Changing - From Tester to Agent Supervisor
- 45:29Advice to Founders: What to Watch Before You Build
related talks
43mTesting Agentic AI Applications: Beyond Traditional QA
19mQA Agents for Android: How AI Is Changing the Game in Testing
44mPanel Discussion | The AI Revolution in Testing: Opportunities and Challenges
35mApp Testing with AI Tools – Make it Easy
29mHow Agentic AI Is Reshaping Mobile App Quality
11mDragonCrawl: Revolutionizing Mobile Testing with GenAI
22mBeyond Bugs: How AI turns QA into Full-Circle Engineers
30mAgentic QA vs LLM Dev: A Live AI Showdown in the DevOps Arena