Seeing the Invisible: Two Years at Wikipedia with W3C's Network Error Logging
summary
Wikipedia deploys W3C Network Error Logging across half of users and receives near-real-time browser error reports (often under a minute, sometimes 0 ms) via a pipeline that posts JSON to Kafka, tags IP/location, and pages on select signals. Concrete wins: 5% sampling avoids overload, TCP timed out and TCP address unreachable trigger actionable pages, a PDF truncation bug was traced to Varnish nuke limits, a San Francisco fiber conduit outage was detected seconds after failure, and an Italian vendor misconfiguration was proven with NEL logs.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 6:03Invisible internet outages
- 9:12Network Error Logging with W3C
- 17:48NEL reports and limitations
- 21:41PDF corruption detection case study
- 22:46San Francisco fiber cut detection
- 24:31DDoS scrubbing provider issue
- 26:39Future NEL plans
- 29:47NEL observability closing
related talks
37mWhat Happens When You Type en.wikipedia.org?
20mNetwork Error Logging: Client side measurement of end to end web service reliability
40mLog Story Short: Chopping Through Forests Of Data
34mFinding the Capacity to Grieve Once More
34mExpect the Un-expected: How to Handle Errors Gracefully
30mOff the Beaten Path: Moving Observability Focus from Your Service to Your Customer
48mMake Sense of your Logs: From Zero to Hero in less than an Hour!
40mWhispers in Chaos: Searching for Weak Signals in Incidents