Self-Destructing Feature Flags
summary
Deploys include a runtime guard that watches errors and flips flags off automatically: a Ruby/Go prototype uses Redis-backed counters to track checks and failures, a 5% failure threshold plus a configurable minimum sample (avoid flipping on bad luck), batching writes (flush ~100ms) and periodic counter resets to keep windows recent, and posts a Slack notice when it disables a flag. The implementation works at modest scale but needs tuning for endpoints that hit 100k checks/sec or long-running beta users.
AI-written summary · about
chapters (28)
- 0:00Open Access Sponsor BLAMELESS
- 1:18Deployment != Release
- 1:55KeyValue Store
- 2:59Enable Globally
- 3:14Enable a percentage of time/traffic
- 4:02Enable for a subset of users
- 4:33Escape Hatch
- 5:12Bugfix Deploy
- 5:55Rollbacks
- 6:27Disable Feature Flag
- 7:24Scenario
- 9:35Remediation
- 10:52HOWTO: Initiate self-destruct sequence
- 13:21CHECK_FOO - FeatureCheck.new failure_threshold: 0.05, minimum: 1_000, do FEATURE_FOO.disable end
- 20:20Atomic Updates
- 20:40Account for volume
- 21:10Distribute keys
- 21:32Batch updates for checks and failures
- 21:54Reset periodically
- 23:33Tradeoffs
- 23:52Difficult to generalize
- 24:36Long-running feature flags for beta groups
- 25:21Requires up-front thinking
- 25:53Acceptable error rates
- 26:04What are your failure modes?
- 27:01How quickly should it react?
- 27:39What units should be used?
- 28:08What if you're wrong?
