Gremlins Exposed: Shining a Light on Mischievous Systems
summary
Shows a practical, repeatable triage: use a concise slip book or A3 problem document to capture clues, start top‑down with HAProxy logs and pipes, spot a saturated HAProxy queue (~1,702) and a single outlier IP, apply a quick rate‑limit and reload to restore performance, then follow up with capacity changes (add server) if needed. Deeper checks use procfs, iptables, trace/curl timing and perf top to decide whether the app or infra is at fault; the quick fixes here work and reduce urgency.
AI-written summary · about
chapters (7)
- 0:00Start
- 1:26Understanding the Problem
- 3:18Gremlins are unreasonable
- 4:54Methodologies
- 17:00Invisible Gremlins
- 19:45The most efficient use of your time is to not spend it
- 21:13Reflection
related talks
39mHow to Use Common Investigation Practices in Troubleshooting
18mObservability tips with HAProxy
31mDebugging the Routing Tier
53mDebugging Go Applications
20mUsing Monitoring to Speed Up Your DevOps Journey
20mProduction is Priority - Self Fix / Heal Techniques
01h 00mSystems @Scale Summer Q&A Week 4
37mAutomatically Detect the Top Performance & Scalability Issues in Distributed