Automated Testing at Scale in Sea of Thieves
summary
Rare builds an automated pipeline on Unreal’s Automation Framework (FAutomationTestBase, IMPLEMENT_ macros, latent commands) plus custom fixtures and networked Blueprint helpers to run over 100,000 tests (23,000 code/actor tests + 81,000 asset audits). CI (TeamCity) runs suites on preview builds, TeamCity runs tests every 20 minutes, and gating requires a green remote run before check‑in. Map/Blueprint tests run about 20 seconds vs 0.1s for coded tests*, and this stops regressions, keeps bug counts low, and sustains rapid releases.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:59Sea of Thieves
- 1:59In this Presentation
- 2:55Why Use Automated Testing?
- 4:21What is an Automated Test?
- 5:49Unreal's Automation System
- 9:48Running Automation Tests
- 12:14Map Tests- Capabilities
- 13:24Map Tests-Challenges
- 15:12The Four Horsemen of the Testing Apocalypse
- 16:28Expanding Coded Tests
- 18:11Test Types
- 19:24Automated Tests Snapshot
- 21:53Our Process
- 31:38Best Practices • Test behaviour, not implementation
- 38:14Summary
- 39:02Automated Testing Is Not a Silver Bullet!
- 39:37We Are Hiring!
related talks
30mHarnessing the Unreal Engine Automation Framework for Performance Measurement
44mTaking CI and automated testing seriously
41mThe Unbreakable Team: Maintaining Build Stability in Unreal Engine
44mHow Sprocket Games Achieved 10x Playtest Velocity for an Unreal Engine Game
20mCI/CD Infrastructure for Efficient Mobile Game Development
55mSo You're Trying to Support a Medium Sized Studio....
32mGame DevOps: Elevating Your Unity and Unreal Build Pipelines With TeamCity
26mThe Impact of Testing on a DevOps Pipeline