Harnessing Cascading Timeouts for System Resilience
summary
Cascading timeouts require upstream calls to have longer timeouts than downstream ones, with a disciplined decrement such as a 500ms buffer per layer to avoid inverted timeouts and resource exhaustion. Example stacks: load balancer 3000ms / backend 2500ms, API gateway 2000ms, app 1500ms, DB with statement timeouts at the connection pool (Hickory) level. Add one retry with exponential backoff, circuit breakers, error-response caching and monitoring; measure, chaos-test, and iteratively tune values from real production latency and incident data.
AI-written summary · about
chapters (9)
- 0:00Introduction and Speaker Background
- 0:34Importance of Timeouts in Distributed Systems
- 2:35Common Timeout Anti-Patterns
- 3:52Principles of Cascading Timeouts
- 7:06Implementing Cascading Timeouts
- 9:33Special Considerations for Machine Learning Pipelines
- 10:34Recap and Key Takeaways
- 12:18Implementation Playbook
- 13:31Final Thoughts and Conclusion
related talks
15mBoosting System Resilience: Exponential Backoff
35mDesign patterns for application resiliency
32mHow does your application behaves when everything goes wrong?
27mWhat's the cost of a millisecond
57mResilience by Design
38mResilience patterns in practice
21mThe Multi Layered Cake of Resilience
42mDesigning for failure