Low tester participation is a source of frustration for many Customer Validation professionals. You’ve invested hours into crafting creative test topics, and you’re following best practices for a comprehensive and approachable communication strategy. But somehow, only a fraction of your testers are submitting feedback. Where did it go wrong? What could you do differently?
Sometimes, it’s not you — it’s your testers.
Like calling your dry cleaners to order a pizza, trying to solicit data from incompatible testers won’t get you very far. Bringing the right testers into your project increases both the amount and quality of the feedback you pull in during Customer Validation. And by the “right testers,” we mean testers who are both in your target market and are enthusiastic about helping you improve your product. Keeping testers engaged throughout your CV test — and securing the results that make your project a success — hinges on recruiting testers with both of these critical traits.
How Recruiting Affects Engagement
Simply put, when your testers aren’t members of your target audience, they’re less likely to use your product and provide feedback. Even if they are submitting issues and ideas, this misalignment can ultimately derail your efforts with input that isn’t relevant to your product’s success.
Centercode Product Director Austin Meyer often cites an example from early in his CV career.
“About five years ago, we recruited users to beta test a mobile phone that offered a physical keyboard as a key selling point. It didn’t take long to realize we had the wrong testers when the majority of incoming feedback said, ‘I don’t like phones with physical keyboards.’ In the end, we had to update our recruitment to capture accurate target market feedback. It was definitely a humbling experience.”
There is a similar effect when your testers don’t have a genuine interest in helping you improve your product. Testers who come into your project half-heartedly will be reluctant to put forth any effort, making it difficult for you to get the data you need. These two traits work together because target market testers have an organic reason for using your product. They’re more likely to be excited about testing it because it’s been built with their needs in mind. That’s why it’s much easier to keep them using it and providing useful insights throughout your project.
Like developing a clear communication strategy, bringing the right testers into your project is a core tenet of high engagement and quality feedback. Let’s take a look at the four most common warning signs that your testers aren’t a good fit for your CV test.
Incompatible Testers: Warning Signs
- Testers don’t/wouldn’t use key product features. It’s more difficult to engage testers on product areas they say they don’t or wouldn’t use. On top of that, any insights they do provide aren’t as relevant because their natural behaviors don’t line up with those of your target audience.
- Testers aren’t completing all their test activities. Testers who continually fail to complete activities are either showing that they wouldn’t naturally use your product the way a member of your target market would or they’re not excited about providing their feedback in the first place.
- Consistent communication doesn’t increase the quality or amount of feedback. Testers who aren’t from your target market are naturally less excited about testing your product, which means they’re more reluctant to give useful insights — no matter how well you’re communicating.
- There’s too much impractical or irrelevant feedback. If the majority of the feedback you’re receiving is impractical or irrelevant (e.g., Austin’s story about the physical phone keyboard), your testers’ attitudes or interests may not match what your product actually delivers.
Quick Fixes for Incompatible Testers
Your recruitment strategy plays a pivotal role in attracting testers who will stay enthusiastic and engaged throughout the course of your project — and screening for testers who don’t fit the bill. While fine-tuning the perfect strategy for your CV test takes some experimentation, here are some quick fixes for the warning signs listed above.
- Check Your Recruitment Strategy. If your testers say they don’t or wouldn’t use key product features, check your recruitment strategy. Did your invitation include a clear product description? Did the questions in your qualification survey help you identify enthusiastic applicants and select the ones whose demographic and technographic traits align with your target market?
- Check Your Communication Approach. If your testers aren’t completing their test activities, you may need to go the extra mile to get them going. Are you quick to follow up on their questions or comments? Are you able to offer any additional, easy-lift incentives for their efforts, like special recognition or swag?
- Check Your Feedback Submissions. If your testers are submitting too many impractical or irrelevant ideas, look for ways to shift their mindset into more of a bug-hunt. Are you able to direct incompatible testers to test product areas that need more bug reports than ideas? Can you identify testers who are submitting quality feedback and nurture them?
Engaging Your Testers Step by Step
Now that you have fresh insight into the relationship between recruitment, engagement, and quality feedback, dig deeper with a phase-by-phase guide to tester participation. Our brand new ebook gives you a tester journey map and tactics for tackling your most persistent participation challenges — from recruitment to test closure.
