My Service Runs at 99.999%...All Those Tweets about Outages Are Not Real...
summary
Claims that a service runs at 99.999% while users hit errors are dissected with concrete failures: a Cortana UI shows an empty canvas not logged as failure because every component is optional so the UX records no error; an edge/CDN layer is blackholing traffic so app‑level logs show normal availability; an experimentation config causes user errors and drives a build of auto‑ramp and emergency shutdown that cuts 95th‑percentile shutdown from ~8 minutes to ~2 and forces better traffic, logging and bot‑filtering practices.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:40Availability
- 5:03MT Pizza
- 12:20Every outage is a gift
- 17:50Outages are easier to handle
- 19:43Data hygiene
related talks
39mOops. I broke the Google, now what?
28mReliability Lessons from Running High Traffic Services for More Than 10 Years
15mShit, our Site is Down! Let's build a culture of reliability
49mWhat It Really Means for Your System to Be Available
33mFrontend Resilience
29mAchieving 99.99% uptime with Open Source Observability
22mDon’t wait for an outage to improve your reliability
22mWhy is it always DNS, TLS, and bad configs?