Down The Rabbit Hole: Chasing bugs from the browser to the backend
summary
A Node.js/browser Jabber-style client built on Socket.IO repeatedly drops connections because an old websocket heartbeat timer survives a reconnect and later kills the new socket, producing ~30 disconnects/minute (about 10% of users); reproduction uses unplugging Ethernet and iptables attempts, inspection uses Chrome DevTools websockets and Wireshark, and shotgun debugging with breakpoints in Socket.IO finds and wraps the stray heartbeat. Deploying the guard collapses disconnects from ~30/min to ~1/min on a service peaking at 900 concurrent connections.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:21Down the Rabbit Hole: a story about chasing a bug
- 2:10my codes are perfect
- 2:50Flaky Internet?
- 3:12Flaky Internet!
- 3:29Lesson: Listen to your users
- 3:44Act II: Acceptance
- 5:18Lesson: Graph all the things!
- 9:23Act III: Repro
- 9:52iptables --help
- 10:58Act IV: Diagnosis
- 11:57Bringing out the big guns Wireshark
- 13:47Shotgun Debugging I stuck breakpoints
- 15:38Act V: The Fix
