Harnessing the Unreal Engine Automation Framework for Performance Measurement
summary
Dovetail Games uses the Unreal Engine Automation Testing Framework plus a C# in‑house automation tool to run nightly FPS, frame‑time, memory and load‑time tests across PS4, Xbox and PC, aggregate results into HTML dashboards (Google Charts) and heatmaps, and track regressions across builds; a scripted pipeline with unique test IDs and latent automation commands records data, reproduced a rare concurrency crash by automated teleporting, and scales toward Gauntlet migration while remaining robust to console OS quirks.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 7:36Writing Automation Tests
- 8:54Simple Automation Test
- 9:51Coding the Test
- 12:34Test Definition
- 13:00Subclass
- 13:17Variables
- 13:31Get Tests
- 14:22Run Test
- 14:54Run Game Binder
- 16:11Copy Results
- 16:51HTML Generation
- 19:58Test Failures
- 21:25Exit Commands
- 22:32Demo
- 22:57Fixed Location FPS Overview
- 23:41Test Target FPS Overview
- 24:22Test Target FPS Data
- 24:51FPS Data Popup
- 25:45Takeaways
- 27:35What we are working on
- 28:15Our current solution
related talks
40mAutomated Testing at Scale in Sea of Thieves
55mSo You're Trying to Support a Medium Sized Studio....
44mPanel - UE4 Pipelines, Teams and Tools
41mThe Unbreakable Team: Maintaining Build Stability in Unreal Engine
43mInside Real-time Production: The Tools that make it Tick
18mAutomated testing
44mHow Sprocket Games Achieved 10x Playtest Velocity for an Unreal Engine Game
54mHow to Apply the Software Engineering Process to the Gamedev Process