Consistent tester participation is the bane of most beta tests. Beta tests most commonly result in somewhere between 25% to 50% participation rates, meaning that only about one quarter to half of the beta testers participated to even the bare minimum expected of them, generally simply walking away with the product without providing any feedback at all. This is a huge waste of both time and money. Thankfully, there’s a number of simple things that can be done to raise these rates significantly.
1. Set Clear Expectations
Many beta tests fail simply because beta testers lack clear guidelines on what’s expected of them. Without these, they’ll likely have no idea what they’re supposed to be doing, with many simply giving up as a result.
Take the time to document and share your objectives and expectations with any customers interested in participating prior to selecting your beta testers. This will ensure that prospective testers understand the commitment required, and have the basic bandwidth and energy available to perform effectively. Once you’ve selected your testers, kick off the project by clearly reiterating these requirements via email.
2. Select Great Testers
This may seem obvious, but it’s something many people simply overlook, or just don’t know how to accomplish. We define great beta testers as those that both match the target market of your product and consistently contribute relevant feedback throughout the entire beta test. Having great testers from the get go is by far the simplest way to reach high participation rates.
The best way to select great testers is by simply asking the right questions.
3. Make it Easy
Removing friction from the beta tester experience is one of the best ways to increase participation. If the act of participating is a chore, or is in any way confusing — you’ll quickly lose the attention of your participants, and your entire test will fall apart.
For very small and short tests, e-mail might be the easiest way to enable easy participation. For larger tests or ongoing beta programs, a dedicated system (like the Centercode platform) will likely make a lot more sense. Not only will this offer a persistent portal for bi-directional communication, but it will also avoid their need to manage multiple logins and user accounts.
Offering testers direct access to your defect tracking system (Jira, Bugzilla, etc) is a bad idea. These systems are far too technical for most customers, and also blurs the purpose of these internal systems (which should be tracking valid and unique bugs).
4. Show Your Appreciation
Beta testers who feel their efforts are appreciated are much more apt to continue providing great feedback. If you’re reviewing their feedback daily (and you should be), it only takes a few extra moments to reach back and thank them for the effort. Please do it.
But don’t overdo it. Testers will know the difference between real gratitude and fake appreciation (e.g. canned responses), and don’t need to be inundated with artificial praise. Just a simple thanks and occasional recognition (even to the whole group) are all that’s needed to help your tester understand how much you appreciate the effort.
5. Keep After Them
If your participation is weak in general, or noticeably faltering — react. It’s important to monitor your testers’ activity on a daily basis. If your testers aren’t contributing, first e-mail them to ask how it’s going, and reiterate the guidelines and goals of the test. If that doesn’t turn them around after a day or two, pick up a phone and call them. While this is time consuming, the value it can provide in a quick participation uptick is well worth it.
When calling testers, avoid showing signs of frustration or annoyance, but rather concern and interest. Ask if they’ve been able to use the product successfully and confirm that they understand the test parameters. Talk to them about their experience so far, explain how important this is to the success of the product, and how grateful you are for the help. Also let them know that you’ll check back in a week (you probably won’t have to).
There’s also a good chance they’ve actually used the product, but simply haven’t reported their feedback yet. Reaffirm that your product can’t benefit from their efforts without that feedback, and you’re looking forward to seeing it personally.
