Preface: We’re assuming that you’re recruiting via an online form, survey, or similar tool that allows you to query your applicants. As long as you have the ability to add a couple of free-form text entries, you’re all set.
The two questions we recommend asking every beta applicant are simply:
- Why do you want to beta test this product?
- What makes you a great beta tester for this product?
That’s it. The answers to those questions will get you about 80% of where you need to be to identify the top beta candidates from any group. The basic idea is that the effort and style put into these answers is an early indicator of how effectively that person will perform as a tester, allowing you to filter the best and worst candidates early, with minimal effort.
Question 1: Why do you want to beta test this product?
- Why you ask: Many people believe that beta testers participate just for the incentives (e.g. free product). While this is true of some, it’s not the case for most — and filtering those reward-focused candidates out of your pool is essential.
- What you’re looking for: General enthusiasm for your product and/or company. For many products, you’re looking for people who are trying to solve a problem they currently have. This is especially true when it’s a new version of an existing product. Above all, if they mention an incentive in their answer – they’re out (although the honesty is appreciated!).
Question 2: What makes you a great beta tester for this product?
- Why you ask: It’s important to identify how the candidates see themselves. Ego is a bad sign, and generally an unfavorable trait in a beta test. These types of personalities can turn off other testers, poisoning participation as a whole.
- What you’re looking for: Creative applicants who answer inventively and go beyond listing previous beta tests they’ve participated in. Applicants that clearly represent the target market of the product should lead the pack.
What you’re looking for from both answers:
- These answers should never be required. Users who decide to ignore either or both of them are simply out of the pool.
- Spelling, Grammar, and Punctuation all count. These devices make both skimming and reading easier and faster, and nothing is more important when reviewing countless bugs, suggestions, forum posts, and survey replies. In other words: ALL CAPS? OUT. no caps? out. No punctuation out. Bed Spllng. Owt.
- There should be some substance in their answer. 1-2 lines can be a strong answer if they’re a great writer, but in general, a solid paragraph of 3-5 lines indicates that they’ve included enough thought and detail to take the question seriously.
- Too much text can be a problem, but not always. If it’s well written, fits the context, and gets a solid point across – that’s fantastic. If it’s 14 paragraphs but could easily be one…move along.
- Make sure they actually answered the questions. While this may seem obvious, you’re looking for more than style here. These questions provided your applicants with their first opportunity to show you that they can clearly follow directions — a key attribute in a beta test.
- While critical thinking and creativity may be hard to identify in such simple questions, there will be times you’ll notice it. Move them to the head of the pack.
- Honesty is hard to identify on a beta application. These questions can help, as once you’ve narrowed your candidates down, it’s useful to ensure these answers align with other demographic and technical questions you’ve asked.
- Most importantly, each question ends with “for this product”, aiming to rein in their answers to the context of your beta test. Pay special attention to those that latch onto that, and focus explicitly on the things relevant to the specific product they’re eager to test. These are good candidates, as they clearly demonstrate their place within your target market.
While reading hundreds of individual free-text answers may seem inefficient (and potentially frustrating and/or dull) at first, these questions offer an enormous amount of insight in a very small space. Remember that regardless of how well the product being tested performs, it is the performance of these people that will determine how successful the beta test of that product is.
We’d love your feedback on these questions, as well as any others you feel would be useful.
