User testing in production: how to run a public beta
summary
Case study: Anvil runs public betas (one editor spent about a year in beta) using a discourse forum for discussions, a dedicated feedback form and email, short paid user interviews (eg $50 credit), and a docs “toggle” implemented with Hugo that serves both versions. Feedback is triaged into five buckets (broken, weird, opinion, feature request, more please), tracked against proxies like outages and GitHub backlog, and communicated back; a 148‑post thread triggered a changelog/newsletter and the beta moves to GA after months of iteration.
AI-written summary · about
chapters (9)
- 0:00🚀 Build Stuff Conference – Get Your Tickets!
- 2:25Introduction to Public Beta Testing
- 6:36When and Why to Run a Public Beta
- 12:11Documentation Best Practices for Public Betas
- 18:00How to Collect and Categorize User Feedback
- 24:16Understanding User Expectations in a Beta
- 30:24How to Manage User Trust and Feedback Effectively
- 36:03Wrapping Up a Public Beta and Moving to General Availability
- 41:11Closing Thoughts: The Importance of Continuous Improvement
related talks
48mUser testing in production: how to run a public beta
47mUser testing in production: how to run a public beta
32mZero to Test, How to Run Your First Beta Testing Program
23mAPI users' feedback: why and how you get it
51mPlaytest-A-Go-Go, Baby!
52mBuilding an A/B Testing Framework that works for Developers
39mQuest for the Holy Grail: Turning User Feedback into Meaningful Change
29mManaging User Feedback