In a very fundamental way, feedback is the lifeblood of a beta test. If your beta test doesn’t garner good feedback it’ll be considered a failure. However, if you can collect a lot of relevant feedback your beta test will be a resounding success. Knowing the best processes to collect the right feedback from your testers is arguably the most important skill a beta manager can have. That’s why we wrote this blog post, to help you know and understand the different kinds of feedback you can collect during beta so you can best leverage them in your test.
In a beta test, there are two kinds of feedback: ongoing feedback and directed feedback. Each type serves a distinct purpose. Their meanings are similar to how they sound. Ongoing feedback occurs naturally throughout your test. It’s comprised of the ongoing insights, responses, and information that your testers report as they use your product. Directed feedback is the information or tasks you specifically request of your testers at different points during your test (such as after installation and at the end of your test).
Ongoing Feedback
Much of the feedback you’ll receive from your testers will be in the form of ongoing feedback. In general, ongoing feedback is circumstantial. A tester will use your product and then submit information based off of their experiences with that product throughout the test. There are four different types of ongoing feedback:
Bug Reports
Bug reports are part of nearly every beta test. By allowing testers to report bugs as they find them you can gauge and improve product quality as well as compatibility among platforms. It’s the best way to get feedback on problems that need fixing, as well as test the real-world performance of your product in a variety of environments and use cases.
Feature Requests
By letting testers use your product the way they naturally would, they’ll come across features they’d like to see in your product. These feature requests can help shape your product roadmap by giving you insight into what features might be added in the future.
Private Journals
Giving your testers the opportunity to write down their general impressions on their experience with the product can provide unique insight into the user experience, while encouraging participation. Private journals are a great way to test usability, gauge the day-to-day user experience, and get a sense of the overall temperature of how your testers view the product.
Discussion Forums
Discussion forums give testers a place to discuss other topics related to the product and collaborate on their feedback throughout the test. These forums can generate all sorts of interesting and insightful peer discussion that might not come up through other methods of feedback collection.
Directed Feedback
Directed feedback, on the other hand, allows you to focus testers on specific objectives you want to achieve during your beta test. By assigning testers to do certain activities or give their opinions on specific subjects you can ensure that you receive exactly the feedback you need from your audience. There are many different activities you can ask your testers to do, but the three most common types of directed feedback are:
Surveys
By sending your testers questionnaires at specific points during your test, you can collect more focused information about everything from the installation/setup process to the user interface to your marketing messaging. Good surveys are the number one way to get your testers’ opinion on specific topics that align with your goals.
Tasks
Throughout your beta test you’ll want your testers to use your product in different ways or give feedback on specific features. By assigning tasks to your testers you can get feedback on very specific actions and outcomes. This is a great way to make sure all the different parts of your product are being tested, or to get testers to help reproduce and regress issues.
Mock Reviews
By requesting that your testers submit mock product reviews during your beta test (similar to those found on Amazon or the App Store), you can get a sense of how well your product is going to do once it launches. They can also give you insight into which features your customers will like and dislike the most, allowing you time to adjust your development and marketing priorities accordingly.
Both ongoing and directed feedback play a fundamental role in your beta test, and when used strategically they can be combined to give you a far better picture of the state of your product. It’s important to remember that different kinds of feedback collect different kinds of information and therefore achieve different objectives.
For example, if you only collect bug reports and exclude feature requests, you might learn a lot about the bugs your product has, but little about what your testers think is missing in your product. It’s limitations like this that either mean missing out on important feedback that you could be getting, or muddling your data because testers are still giving you that feedback but through the wrong report forms.
By using a combination of ongoing and direct feedback techniques, a beta manager can collect the variety of objective feedback needed in order to make the final improvements that will ensure a successful product launch. Learn how to collect and manage high quality feedback with our Feedback Playbook!
