The relevancy of your feedback is one of the most crucial ingredients to your Beta Test. This means your tester team is the determining factor for its success or failure. For feedback to be relevant to your objectives, it has to come from true members of your target market.
In Part Two of Finding Quality Beta Testers, we’re focusing on who your target market is relative to your Beta Test, how many testers you need, and how to determine the right targets for your recruitment.
Defining Your Target Market
Testers who aren’t in your target market can provide lots of feedback, but their attitudes and behaviors won’t reflect those of your actual customers. To ensure the success of your Beta Test – and your product – it’s essential to target your recruitment towards the very people whose needs and desires match your target audience. At the same time, recruiting testers that fit extremely specific traits can become very challenging very quickly.
Your product’s features are solutions to your target market’s problems, but every user won’t have the same problems. It’s very rare for users to use every feature a product has to offer. We often see Customer Validation teams struggle to find testers who represent the entirety of their product’s audience, then bend over backwards to get a handful of testers to evaluate tons and tons of features. What you want to identify instead are certain testers who are willing and capable of accurately evaluating the certain features that align to their attitudes and behaviors.
A helpful technique is to identify key “must-have” traits for testing a certain feature. These are your “core requirements” – the baseline qualifications every tester needs to produce relevant feedback. Your core requirements provide a clear outline for who you’ll be targeting during recruitment and which individuals are eligible for consideration for your test opportunity. Once you have a group of testers that meet the basic qualifications, you’ll then add additional criteria (only as needed) to meet specific goals.
You probably have a good idea of your target market already, but it never hurts to reach out to your product or marketing team for detailed information about your audience. Work with your stakeholders to prioritize the key characteristics that will be reflected in your core requirements.
Your “must-have” and “nice-to-have” tester traits will fall into one of three categories:
Demographics
Most products target individuals based on consumer or business-related demographic traits. These demographics include qualities such as age, gender, education, marital status, occupation, economic status/income, home or vehicle ownership, household size, type of dwelling, region, and the list goes on. If your product is targeted at businesses, relevant demographics include vertical, company size, organizational structures, etc.
Technology
In today’s interconnected world, your product’s performance will likely be impacted by the technical environments where it’s used and the products it interacts with. With this in mind, you’ll want to recruit users with as wide a range of relevant technology products and environments as you reasonably can. This includes factors such as internet speed, wireless capabilities, computer operating system, automobile type, or type of appliance.
Knowledge
This refers to the specific experience levels or skills (like familiarity with testing, technical prowess, or occupational expertise) your ideal testers should have. It’s common for technology companies to develop products that reach markets with diverse groups of people, including people with less technology experience and/or tech savants. Determining the knowledge-based traits of your target audience and recruiting test participants who share similar skills and experiences will help you achieve true Customer Validation for your product.
Now that you have a good feel for who you want to recruit, let’s determine how many testers you need to yield actionable feedback.
How Many Testers Do I Need to Recruit?
As a rule of thumb, you’ll need about 75-125 testers for a Beta Test. But – and here’s where it gets tricky – the actual number of testers you need changes depending on other factors.
Your goals, your product, logistical constraints (like the number of testing units available), and the expectations of your organization all contribute to the size of your ideal tester team.
How Your Goals Affect the Size of Your Tester Team
As we mentioned in the first part of this series, your recruitment strategy always ties back to getting quality feedback that helps you meet your goals. If your goals require 5% margin of error with 95% confidence interval, then you need to have the right number of testers to support that. However, if your goal is to validate stability in 10 unique environments over 2 weeks, it requires a lot fewer testers.
How Your Product Affects the Size of Your Tester Team
The number of units you have available, the number of features you need to test, and the complexity of those features can all influence the number of testers you need. As the number of use cases for your product increases, so does the necessary size of your tester team. On the other hand, your number could also be limited by the availability of test units – you may not have 75 pre-release units ready to go. Negotiating what’s necessary with what’s feasible is critical.
More isn’t always better. With most tests, you will reach a point of diminishing returns even before you’ve managed to recruit hundreds of testers. With too many testers, you’ll likely discover redundancies in your data and repetition in the insights testers deliver. The key is to recruit enough testers to achieve a good level of confidence in your results, but not so many that you create a ton of work for yourself through inefficiencies. You can read more about diminishing returns in the Beta Tester Recruitment Kit.
How to Identify Your Ideal Tester Team
- Single out one or two important technical aspects and a couple of demographic characteristics that you’d like to focus on. For example, you might want to focus on the user’s gender and parental status (demographic), as well as their level of computer expertise and their desktop operating system (technical).
- Define the specific traits you want reflected in your tester team. For example, a certain ratio of male to female users, parents to non-parents, Windows to Apple users, and tech gurus to those with limited computer knowledge.
- Using your core requirements as a basis, zero in on the additional characteristics of your target audience and use them to separate your tester team into clearly defined groups (segmentations). Once you’ve defined your demographic and technical segmentations, each one will becomes a bucket that each individual tester can fall into.
- Start recruiting to fill each of these buckets, mapping the number of testers to the proportional size of your market that the persona represents.
For more tips to clearly map out the ideal target for your recruitment and recruit them to your next project, use the templates and exercises available for free in the Beta Tester Recruitment Kit!
