User testing in production: how to run a public beta
summary
Explains how to run a public beta from a developer-tooling perspective: start when the MVP lets users solve a new problem, keep it opt‑in and reversible, document either inline or with dual toggled docs, and collect unstructured feedback via forums/Discourse, email, interviews and issue trackers (GitHub) rather than rigid forms. Classify feedback into five buckets (broken, usable but painful, UX expectations, feature requests, extra asks), act on fixes, announce changes, track GitHub backlog and outages (Honeycomb) and use feature parity to decide GA.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:11introduction & disclaimers
- 1:11what is a public beta?
- 5:04reversible
- 6:17new experimental features
- 6:51significant changes to core functionality
- 17:56interleave feedback points with product
- 27:32communicate back
- 30:01what can go wrong?
- 30:21ship breaking changes
- 36:57feature parity with the existing product
- 37:27unblocking future development
- 38:19communicate with your users
related talks
48mUser testing in production: how to run a public beta
44mUser testing in production: how to run a public beta
23mAPI users' feedback: why and how you get it
15mPut On Your Product Hat When Championing an Internal Tool
24mUsability Testing for Developers
27mNo product to test yet? No problem.
29mManaging User Feedback
24mEffective Product Experimentation: Using Data-Driven Techniques for Iteration