Creating & Scaling Meaningful Performance Tests in CI

on YouTube ↗38m

" summary

Focuses on testing only controllable, reproducible, impactful metrics (e.g., app startup) and warns against testing raw network latency or dynamic UI without mocking. Recommends combining tracing and sampling: use Perfetto SQL layer to get precise start/stop slices, run simpleperf flame graphs to diff method-level changes, and orchestrate runs with macrobenchmark separate test APK or UI Automator to avoid attaching to the app process. Calls out variance, iteration counts, device choice (Pixel 6 Pro example) and cost/accuracy/speed tradeoffs.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 1:50What is mobile performance
3. 7:07Framework for metrics
4. 13:02Tools for measurement
5. 26:57Testing strategies
6. 33:30Scaling in CI
7. 37:44Q&A and summary

" related talks

• 39mCreating & Scaling Meaningful Performance Tests in CIRyan Brooks · droidcon New York 2022
• 29mProgressive Testing to Meet the Performance ImperativePaul Bruce · DevOpsDays Boston 2018
• 41mShift Left your Performance TestingHari Krishnan · Conf42 Site Reliability Engineering 2021
• 43mThe Perf Testing Trifecta for Optimal End User ExperienceRobert Healey · Testing Talks Conference 2024 Melbourne
• 42mMeasuring Performance with Functional Tests: A Low-Effort Approach for High ImpactPeter Sabev · TestCon Europe 2025
• 41mContinuous Performance Testing in DevOpsLee Barnes · TestCon Europe 2021
• 33mPerformance Testing in Agile and DevOpsGopal Brugalette · TestCon Europe 2019
• 53mEarly Performance Testing: Finding News You Can UseEric Proegler · CAST 2014