12 Factors of Pain and Suffering
summary
A developer recounts concrete failures and fixes: copying one codebase into 20+ spam sites instead of templatizing it, forgetting to install optipng jpegoptim gifsicle on a new node so the image API crashes, hardcoding a Rackspace key and triggering a $25,000 bill from automated abuse, storing voice prompts on single servers so a failover loses user recordings, using sqlite locally and hitting Postgres varchar errors in prod, an autoscaler killing processes that download hundreds of GBs and a log overwrite that costs half a million dollars.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:30Who am I
- 3:07Managing your codebase
- 7:45Dependencies
- 11:02Storage
- 14:12Backing Services
- 18:52Build Release Run
- 21:31Processes
- 23:20Port Binding
- 24:31Concurrency
- 28:02disposability
- 29:29how I screwed it up
- 31:16dev prod parody
- 32:51logs
- 34:03API service
- 36:25Admin processes
- 37:09I screwed this up
related talks
31mPixels, Promises, and Panic - Horror Stories of Production Nightmares
40mAvoiding Pain When Operating in the Cloud
34mNightmare Before Best Practices
42mPlanning To Fail
28mLearning from High Profile Failures
30mAvoiding software fails: few metrics to improve application reliability
27mFAIL: The Best Ways To Bring Down Your Website
32mBuilding for Gracious Failure