How you communicate with your testers has a huge impact on the success of your beta. If you don’t communicate clearly or often enough you risk alienating or confusing your testers. On the other hand, if you communicate too much you could frustrate testers with your constant micromanaging. Both scenarios depress tester participation and limit feedback. We’ve put together some tips to help you find a balance as you talk with your testers, and some of the tips aren’t what you’d expect!
1. Use Email Sparingly
You’ll want to ensure email communication is infrequent, important, and to the point. Consider limiting its use to messages that are either time-sensitive or very important. For other matters, stick with your beta test tools (bug report comments, discussion forums, etc.) as your primary means of communication. Why? High email volume risks frustrating your testers and diluting the perceived importance of your messages. Sending frequent emails also blurs the lines of your communication protocol, encouraging testers to direct communication to you via email instead of your beta test tools.
2. Offer Positive Reinforcement
Don’t forget to praise users for particularly good feedback or participation in the test. Comments like “excellent question” or “great thought” not only make your users feel appreciated, they also reinforce that you’d like to see more of that behavior. After all, it’s sometimes easy to overestimate how much guidance you’ve actually given your testers.
3. Give Preferential Treatment
“Pay extra attention to your super users. Some testers are like part-time employees and will spend hours each day testing, helping others in the forums, and evangelizing your product. Reach out to these testers with a personal email, phone call, or extra gifts to make them feel special. It’s worth it in the end!” -Geoff Griffin, TiVo
4. Enforce Communication Protocol
If you want to keep your beta test data organized, enforcing communication protocol is critical. When participants have general test concerns, like a change in their availability or questions about how to be a better tester, it’s okay for that communication to happen by any means necessary. Where you want to be strict is in how participants give you beta test feedback. If some people use your beta management tools, others send in random emails, and one person calls you to report bugs, things can get out of control quickly.
5. Write for the Appropriate Audience
It’s important to remember your audience demographics when communicating with participants. Take young adult gamers as an example. They’ll tend to speak, engage, and operate a lot differently than older business software testers. That doesn’t mean your tasks have to revolve around frags and pwning, but don’t expect them to provide detailed cost-benefit analyses for your product.
6. Time Your Communications
“If you are running a global program, or even testing across the country, be mindful of when you send the email based on time zones. Depending on the target audience, you may find they have different habits as to when they check email and when they arrive at/depart from work. Don’t be afraid to ask these kinds of questions of your beta participants in exit surveys, and remember that you are asking them to help you free of charge in most cases, so you need to flex to their schedule, not the other way around. Scheduling outbound emails via macros based on time zones can yield a high return rate if done right.” -James McKey, Symantec
7. Click Send
“Don’t be paralyzed by perfection with those emails you have to send to lots of beta participants. You’ll have to send several of them to establish good communication and follow up with your testers. If you brood over the perfect language rather than just using your own casual, engaging tone (assuming you have one), then you’ll likely communicate too little, too late, or in a manner that puts off your audience.” -James McKey, Symantec
8. Stick to Your Word
Keep your promises and watch your words. If you say something during a beta test, your testers will hold you to it. And this applies all sorts of issues, including test schedules, product features, and (perhaps obviously) incentives. You might think testers won’t care if you have to go back on something you’ve said, but you’d be underestimating the personal stake they feel when it comes to testing your product.
9. Keep Things Under Wraps
Now that you know how important it is to watch what you say, what do you do when testers ask hard questions? You still have the option to not say anything. The point of a beta project is to gather tester feedback, so as long as you aren’t ignoring the question or otherwise being rude, you aren’t obligated to give full answers or any answer at all. The key with responding to these questions is to handle it in such a way that you don’t upset the tester and derail future participation.
10. Don’t Announce Upcoming Builds
This one is a cardinal rule of beta testing: never, ever tell participants that you have a new build coming. They will stop testing and wait to see what the new build brings if they know a release is on its way. And if you think about it from their perspective, this makes sense. Why would they keep testing when anything they uncover could already be fixed unbeknownst to them? You see it differently because you probably have the benefit of knowing what is being addressed in the new build.
11. Be Professional, But Be Human Too
Balance professionalism with personality when managing your tests. Not only will this help your participants feel like they are part of a team, but it can also pay dividends when it comes to participation levels. It’s hard for testers to relate to a stodgy beta manager, and if they can’t relate to you they’ll feel less compelled to help you.
12. Don’t Burn Out Good Testers
It’s natural to want to identify good testers and keep inviting them to more tests. However, even active and engaged beta testers can suffer burnout. Keep a careful watch on their productivity if you use them often, so you know when this might be happening. And if you want to keep your relationship going with the tester while still imposing a break, consider using them as an alternate.
13. Consider Home or Office Visits
“Try arranging visits with beta customers in their test environment (often, if you can). Invite product designers along for the trip, too. One of the challenges of product development is verifying assumptions about how your users use your product. What better way is there to do that than seeing for yourself?” -William Marshall, Avid
These nuggets are just the tip of the iceberg. If you’re interested in more ways to encourage high participation in your tests, download our free guide to reaching 90% participation. And if you’d like more tips on running your beta test, you’ll find these and more in our eBook: 100 Tips for Better Beta Tests — just click the button below.
