Creating & Scaling Meaningful Performance Tests in CI
summary
Focus on macro user flows you can control, reproduce, and impact — e.g., a ~4s app startup broken into traced spans — and avoid unmocked network or dynamic-content tests that add noise (one run showed a 1.5s difference from backend variance). Collect Perfetto traces and query them via the Perfetto SQL layer, supplement with Simpleperf sampling for flame graphs to pinpoint functions, and run Macrobenchmark or UI Automator in CI at PR cadence. Trade-offs are cost, accuracy and speed; custom UI Automator setups give flexibility but increase maintenance.
AI-written summary · about
chapters (9)
- 0:00Introduction to talk
- 1:14What is mobile performance
- 6:51Framework for testable metrics
- 13:10Tools for measuring performance
- 26:47Controlling variance in tests
- 27:51Macrobenchmark in CI
- 31:43UI Automator for testing
- 34:28Scaling performance tests
- 37:57Summary and key takeaways
related talks
38mCreating & Scaling Meaningful Performance Tests in CI
40mWhy component level performance testing should always be part of CD and CI
33mPerformance Testing in Agile and DevOps
40mAutomated Performance Testing In Preproduction
41mContinuous Performance Testing in DevOps
29mProgressive Testing to Meet the Performance Imperative
42mMeasuring Performance with Functional Tests: A Low-Effort Approach for High Impact
43mThe Perf Testing Trifecta for Optimal End User Experience