⚡A Resilient Application in a Partially Degraded World ⚡
summary
Describes how 6play improved real availability by adopting concrete practices: define SLIs/SLOs (e.g. <25ms latency, <0.01% errors); use short timeouts and bounded retries with exponential backoff; deploy canaries and feature flags for safer rollouts; run Chaos Monkey-style failures on AWS spot instances to harden autoscaling; prefer asynchronous messaging to reduce blast radius; randomize cache TTLs to prevent thundering herds; and rely on logs/metrics/tracing to trigger actionable automation and load-shedding instead of waking engineers for every alert.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 3:01Availability definition
- 9:26Distributed systems theory vs reality
- 13:17Reduce blast radius
- 14:00Observability and key metrics
- 17:30SLOs and reliability budgets
- 19:59Actionable alerts
- 21:50Predicting failures
- 22:54Short timeouts and resources
- 25:22Feature flags and careful deployments
- 26:47Redundancy and multi-region deployment
- 28:35Async microservice communication
- 30:00Random cache to avoid spikes
- 31:45Spread load with randomness
- 33:54Exponential backoff retries
- 35:41Degraded mode and UX
- 38:28Load shedding
- 39:32Chaos engineering
- 41:18Common goals and simple tech
- 42:43Incident management and communication
- 43:48Conclusion
related talks
52mPerformance & Stability Anti patterns
35mDesign patterns for application resiliency
53mResiliency and Availability Design Patterns for the Cloud
34mToward continuous resilience
55mGoing Resilient
45mA resilient application in a partially degraded world
16mEngineering Failure-Resilient Systems
51mPatterns for building resilient software systems