Applying Friction for Development Productivity with Circuit Breakers
summary
Slack adds circuit breakers into the Checkpoint orchestration layer that defers or sheds jobs instead of executing doomed requests, using programmatic metric queries from Prometheus + Trickster to open circuits and notify teams in Slack and the UI. The scheduler marks jobs complete without running them, cutting CI load growth from ~10% to ~1.5% month‑over‑month (March 2020–2021) and preventing cascading CI failures for two years. orchestration-level circuit breakers provide isolation and faster feedback.
AI-written summary · about
chapters (8)
- 0:00Start
- 4:16Dark age of cascading failures
- 9:33Slack Webapp Architecture (super high level)
- 11:02Circuit Breakers for Checkpoint
- 11:59Connecting the dots (with Checkpoint)
- 13:45Open circuits during Checkpoint orchestration
- 16:29Circuit Breaker: Trickster
- 18:28Circuit Breakers: Load Growth
related talks
10mCombine Support and DevOps work in an Agile team
20mOptimizing Delivery with SlackOps
54mBridging the Gap CICD Mind the Gap
36mDeveloper Productivity On a Budget
19mReducing Context Switching & Unplanned Work Through Platform Engineering
16mShift Left, Not Off a Cliff: Why Your Testing Strategy Is Breaking Developers
20mChaos: breaking your systems to make them unbreakable
05m⚡Lightning Talk: Scheduling Success: Precision Updates for Continuous Manufacturing Operations