Building Koi Pond: Simulating Millions of Slack Clients
summary
Builds a concrete load tester called Koi Pond that replaces thousands of headless Chrome clients with lightweight Go simulators and a keeper/school architecture, letting tests run continuously in production. Claims cost drops from $0.37 per puppet per day to 0.1 cents per koi per day, demonstrates coordinated formations to recreate thundering herds and fans out to 500,000 simulated users, and reports successful prelaunch tests that catch websocket and client-side scaling issues.
AI-written summary · about
chapters (11)
- 0:00Introduction and context
- 1:21Early load testing efforts
- 3:50Architecture and limitations
- 6:15Real-time messaging challenges
- 8:28Scaling for larger customers
- 13:28Building Puppet Show
- 17:32Developing Koi Pond
- 20:24Configuring Koi Pond
- 24:15Simulating complex behavior
- 25:40Demonstration and results
- 30:24Conclusion and takeaways
related talks
31mKill! Kill! Die! Die! Load Testing With A Vengeance
22mBuilding a Flexible, Scalable Infrastructure to Test Dolby.IO at Scale
27mTesting Before You Scale & Making Friends While You Do It
45mSchrödinger's Pokémon: Observability for Chaotic Load Testing
48mHow to not Crash in Prod, Load Testing Crash Course
33mHold My Beer - Load Testing. In Production. On Autopilot.
26mTo see what others can not... You must climb the mountain, A Introduction to Load Testing
38mCloud chaos and microservices mayhem