Production Testing Through Monitoring
summary
Testing alone fails to catch real failures so monitoring must track concrete business and system signals and be iterated. Examples: a social site returns HTTP 200 while tweets silently drop, a game bug becomes an in‑game plague, a company with ~100 million users and ~5,600 metrics traces a revenue dip to an ESP blacklist, and a surge in AmEx declines forces a new check. Build instrumentation for revenue and registrations, alert only actionable thresholds, and correlate business metrics with system metrics.
AI-written summary · about
chapters (14)
- 0:00Production testing and monitoring intro
- 2:29Testing limits in production
- 5:27Edge cases in production
- 8:41Why testing alone is insufficient for production systems
- 10:08Monitoring purpose and scope
- 12:40Aligning monitoring with business
- 14:04Case study: Revenue decline troubleshooting
- 16:52Instrumentation evolution
- 18:17Monitoring vs testing in production
- 20:11Monitoring tools and correlation
- 21:24Alerting thresholds and false positives
- 23:20Monitoring culture and roles
- 25:09Dashboards vs alerts
- 27:01Q&A
related talks
28mProduction Testing through Monitoring
30mReducing Chaos in Production: Testing vs. Monitoring
15mWhy I Prefer Good Testing Over Excellent Testing
49mAll Levels of Performance Testing and Monitoring in Web Apps
53mEarly Performance Testing: Finding News You Can Use
09mMonitoring? Not OPS problem
32mSynthetic Monitoring and E2E Testing: 2 Sides of the Same Coin
58mObserving Applications in Production