Launching Pokémon GO
summary
Niantic launches in Australia/New Zealand then the US ~12 hours later and immediately exceeds capacity, running on Google Cloud with a 5X provisioning cap that triggers repeated quota-driven outages; the distributed engineering team coordinates over Google Hangouts, scrambles to add datastore and compute, and hunts a repeatable failure that collapses the system at a daily 2:30pm cliff tied to East Coast traffic, then fixes it by replacing an exotic open-source component with the standard Java library.
AI-written summary · about
related talks
16mRunning Services at Google Scale
39mOops. I broke the Google, now what?
36mToPlay on Google Cloud
49mServing 30,000,000 Requests an Hour in the Cloud
13mScaling for a Pandemic: How We Keep Ahead of Demand for Google Meet during COVID-19
43mGoogle - Load-shedding
38mLive Ops Disaster Stories: What We Learnt
39mBuilding for the (Inevitable) Next Cloud Outage