Building an 'Oops-Less Operation' at Scale
summary
A practical checklist for keeping services up at scale: design for power and network blackouts, assume human cable mistakes, and require management buy‑in to fund reliability work. Use circuit breakers and strict timeouts to fail fast and route traffic to healthy instances, practice restores not just backups, and enable point-in-time recovery for databases like Oracle or Postgres. Built a microservice‑first monitoring stack (Instana) to observe rapid failures and confirm these patterns actually detect and reduce outages.
AI-written summary · about
chapters (6)
related talks
51mDesigning for Failure: Build Resilient Services
52mPerformance & Stability Anti patterns
10mDesigning for Failure using AWS: Resilient Systems
29mBuilding Resilient Systems
28mBuilding Resilient DevOps Pipelines: Strategies for Handling Unexpected Failures &...
16mEngineering Failure-Resilient Systems
15mDevOps at scale
42mArchitecting for Failures in Microservices