There’s a long-standing practice in Marketing, Product Development, and Usability Testing called “following your customer home”. Intuit’s famous Follow Me Homes are a great example. The practice is also gaining traction in beta testing, so we thought now would be a good time to discuss some best practices and caveats for scheduling meetings with beta testers. Done correctly, these meetings can have a great effect on participation and open doors to insights that are hard to obtain through other methods.

Before we get into specifics, it’s important to understand these meetings can take a lot of different forms. You can have a true in-person meeting in your beta tester/customer’s home and observe the unwrapping, installation, and early user experience. You can also invite testers into your offices and perform observations there (a tactic often used in video game play-testing to prevent leaks). But the low-hanging fruit in this area is using technology to facilitate these meetings online. This is great for software products, but hardware companies face an uphill battle. By using software like join.me (a very simple web application that requires no installation) your beta testers can have screen-sharing sessions with your beta team without the intimidation factor of more common enterprise solutions.

Use Cases and Benefits of Beta Tester Meetings

• Usability and User Experience Observations: This is often the first scenario that comes to mind. The opportunity to see your beta testers using your product live and ask questions will give you insights that they might never think to write about in more traditional feedback forms. You’ll want to be careful not to lead or instruct your beta testers. It’s best to observe their natural behaviors. You can always ask them why they did certain things if you need additional information afterwards.
• Bug Reproduction Observations: Depending on how technical your beta testers are, the feedback you get in bug reports may not be detailed enough to help your QA team. Scheduling a beta tester meeting, particularly an informal screen-sharing session, gives the beta tester an opportunity to show you what they’re experiencing, at which point you can use your technical expertise to help fill in the gaps and really understand the issue. It will help get to the root cause faster than eliciting more written feedback.
• Beta Kick-Off Meetings: We always advocate starting your beta tests off strong. That means clearly communicating expectations, having a well-defined plan with feedback goals, and actively engaging your tester team. If that team of testers is small enough, you might consider individual meetings at the start of testing. Connecting with a real person can greatly increase a tester’s commitment to the project and it’s a good time to cover general expectations (again) and clear up any questions.
• Motivating Non-Compliant Testers: Part of how we maintain 90% participation or higher in our managed beta tests is that we take compliance very seriously. If a tester is not participating, we reach out with an email or phone call to bring them back into the fold. And, truthfully, it works very well. But perhaps adding a wrinkle to this process, by using a screen-sharing session to directly nurture their engagement with the test would work even better. You’ll want to be careful if you find yourself doing this too often. It could be a sign that you’re recruiting the wrong testers.

Caveats

There are two caveats to mention with schedules these types of meetings. First, you don’t want to rely on them too much. If you schedule these meetings often, testers will start to withhold feedback at other times and just wait to tell you verbally. Even if they remember to tell you everything, it creates a major beta management headache.

Second, you may quickly discover that scheduling meetings with a whole team of testers is difficult. Trying to sync the times you’re available with the mutually unknown schedules of your testers is incredibly time-consuming if you’re doing it manually. To address this problem, we’ve added meeting scheduling functionality to Connect 9.4, which lets you list appointment times that your testers can claim when they log into the project. No more calling and emailing to check availability, then cross-referencing alternate times if someone throws you a curve ball.

Learn more during a demo of our software!

Image courtesy of Flickr user Robert S. Donovan.