Additional Resources on Beta Test Recruitment and Motivation

• Two Questions to Ask Every Beta Applicant
• 5 Great Beta Tester Recruiting Sources
• 7 More Beta Tester Recruiting Sources
• 5 Ways to Increase Beta Tester Participation

Planning to launch a beta test soon and need a little guidance?
Download our free whitepaper, Getting Ready for Beta Testing.

Before we get into specifics, it’s important to understand these meetings can take a lot of different forms. You can have a true in-person meeting in your beta tester/customer’s home and observe the unwrapping, installation, and early user experience. You can also invite testers into your offices and perform observations there (a tactic often used in video game play-testing to prevent leaks). But the low-hanging fruit in this area is using technology to facilitate these meetings online. This is great for software products, but hardware companies face an uphill battle. By using software like join.me, which is a very simple web application and requires no installation, your beta testers can have screen-sharing sessions with your beta management team without the intimidation factor of more common Enterprise solutions.

Use Cases and Benefits of Beta Tester Meetings

• Usability and User Experience Observations: This is often the first scenario that comes to mind. The opportunity to see your beta testers using your product live and ask questions will give you insights that they might never think to write about in more traditional feedback forms. You’ll want to be careful not to lead or instruct your beta testers. It’s best to observe their natural behaviors. You can always ask them why they did certain things if you need additional information.
• Bug Reproduction Observations: Depending on how technical your beta testers are, the feedback you get in bug reports may not be detailed enough to help your QA team. Scheduling a beta tester meeting, particularly an informal screen-sharing session, gives the beta tester an opportunity to show you what they’re experiencing, at which point you can use your technical expertise to help fill in the gaps and really understand the issue. It will help get to the root cause faster than eliciting more written feedback.
• Beta Kick-Off Meetings: We always advocate starting your beta tests off strong. That means clearly communicating expectations, having a well-defined plan with feedback goals, and actively engaging your tester team. If that team of testers is small enough, you might consider individual meetings at the start of testing. Connecting with a real person can greatly increase a tester’s commitment to the project, and it’s a good time to cover expectations (again) and clear up any questions.
• Motivating Non-Compliant Testers: Part of how we maintain 90% participation or higher in our managed beta tests is that we take compliance very seriously. If a tester is not participating, we reach out with an email or phone call to bring them back into the fold. And, truthfully, it works very well. But perhaps adding a wrinkle to this process, by using a screen-sharing session to directly nurture their engagement with the test would work even better. You’ll want to be careful if you find yourself doing this too often. It could be a sign that you’re recruiting the wrong testers.

Caveats

There are two caveats to mention with schedules these types of meetings. First, you don’t want to rely on them too much. If you schedule these meetings often, testers will start to withhold feedback at other times and just wait to tell you verbally. Even if they remember to tell you everything, it creates a major beta management headache.

Second, you may quickly discover that scheduling meetings with a whole team of testers is difficult. Trying to sync the times you’re available with the mutually unknown schedules of your testers is incredibly time-consuming if you’re doing it manually. To address this problem, we’ve added meeting scheduling functionality to Connect 9.4, which lets you list appointment times that your testers can claim when they log into the project. No more calling and emailing to check availability, then cross-referencing alternate times if someone throws you a curve ball.

Do you use beta tester meetings to help improve your beta results? Let us know in the comments!

Want to learn about more of the great features in Connect 9.4?
Click here to request a live demo.

Image courtesy of Flickr user Robert S. Donovan.

With 2012 rapidly approaching, we’re releasing another free beta testing resource that we hope will be very helpful as companies get ready to launch new products in the upcoming year. The whitepaper, entitled Getting Ready for Beta Testing, explores tips and best practices that product developers can rely on as they grapple with the question of when to begin their beta test.

In our experience, thorough beta test preparation requires focus in three distinct areas: product readiness, team readiness, and tester readiness. Each area gets a detailed treatment in the whitepaper, helping readers avoid common pitfalls like distributing beta product without preparing for end-of-beta procedures or harming tester participation because of avoidable delays.

You can download the document by clicking the link below. Please share it with others (and don’t forget about our other free beta testing resources). But, before you go, what sort of processes do you undertake before beta testing? Let us know in the comments!

Image courtesy of Flickr user tableatny.

1. Set Realistic Goals

There may be many goals you want to accomplish during your beta, like stressing certain features or testing different teams and resources under live customer action. However, you can only move so many mountains during a single project. If you think of each beta goal as a mini project that requires scarce resources like time and the focus of your tester team, you’ll understand why it’s important to space things out. Generally, we recommend specifying one named goal per week, in addition to basic test functions like validating quality and collecting general product feedback. If you try to accomplish several major goals at once, you risk making little progress with any of them.

2. Balance Your Core Parameters

There are three core “moving parts” in every beta test: (1) the size of your beta tester team, (2) the duration of your beta test, and (3) the set of specific goals that you’re trying to achieve. It’s useful to think of these resources in equilibrium, where an adjustment to one has a ountervailing effect on the others. Use this to your advantage in planning the most effective test. For example, increasing your test duration will allow you to accomplish more goals. If your schedule gets cut, you can often compensate by adding more testers and still achieve your goals. Factor these three parameters into your planning, but also keep them in mind when unexpected events require you to make adjustments throughout your beta.

3. Size Your Tester Team Based on Target Markets

Most beta tests introduce a product to numerous target markets (or market segments), typically based on attributes such as region, gender, income, and technical knowledge or requirements. It’s important to keep in mind that the number of market segments you need to reach should directly increase the size of your tester team. You don’t want to work in the other direction and select a number of testers to recruit, then hope you’ve adequately covered your target market. If the composition of your tester team doesn’t bear an accurate relationship to desired market segments, it’s difficult to weigh the relevance or importance of survey results (i.e., they become anecdotal).

4. Don’t Forget About Ramp-Up Time

If you’re starting a beta program from scratch, recruiting a great tester team can easily take two weeks or more, depending on your target market requirements and the size of your test. If you’re starting a beta project with either an existing (hopefully interested) customer list or an established beta community, ramp-up can be reduced to only a few days. Either way, it’s important to include this period in your plan. The last thing you want is to sacrifice planned testing time to make your product release window because unanticipated recruitment delays consumed 25% of your beta testing schedule.

5. Plan for Idle Participants

It’s extremely uncommon for every beta tester to meet the goals you’ll set for them. Sometimes participants are simply unmotivated, but many times other personal or business responsibilities take precedence. It’s crucial to factor this into the recruitment section of your beta plan. If you’ve never managed a beta test before (therefore having minimal recruitment and participation management experience), you should plan to include at least two to three times the necessary participants to meet your goals.

From what we’ve seen, the relationship between goals, testers, and test length is one of the most overlooked, yet critically important, considerations in beta management. If you master it, you’ll be way ahead of the curve. So keep these tips in mind as you plan your next beta—they’ll help you better accomplish your goals in the finite time available.

How do you approach planning your beta? Please leave a comment!

Looking for more tips to make your next beta great?
Click here to download our free eBook.

Image courtesy of Flickr user Ernst Vikne.

A detailed beta test plan is one of the key components of a successful beta test. When your beta test is well-organized from the start, it allows you to stay focused on your product during testing, rather than playing catch up. To help more people develop great beta testing plans, we decided to create free beta test planning kits. We started with a hardware beta test planning kit, and today we’re publishing a kit for software betas that caters more to software-specific topics like digital distribution and product keys. So whether you’re running a desktop, mobile, gaming, or web beta, our beta test planning kit can help you do it better.

Our free Software Beta Planning Kit includes everything you need to craft a rock-solid beta test plan. Here’s what you’ll find inside:

1. Comprehensive Beta Test Plan Template

We’ve developed a beta test plan template that’s complete, easy to work with, and will make you look great. Our template covers more than 20 unique beta planning topics from scheduling, to budgeting, to closing your beta.

2. Helpful Companion Guide

We’ve included a detailed guide that walks you through every topic found in the template. The guide also includes more than 25 great tips to help build a bulletproof beta.

3. Complete Sample Beta Plan

Lastly, we’ve included a working example of a complete beta testing plan using our template. If you’re looking for inspiration as you’re drafting your own beta plan, this is your handy reference.

Download our Free Software Beta Test Planning Kit Now

Like all projects, a successful beta test begins with a well-developed plan and concrete objectives. Our objective is to make that planning easier by offering resources built on our 10+ years of experience in beta management. One important lesson we’ve learned is that creating a hardware beta plan is decidedly different from creating a software beta test plan. There are subtle nuances in areas like budgeting and product distribution and completely new problems like acquiring beta hardware and replacing defective units. We’ll help you prepare for all of them.

Our free Hardware Beta Planning Kit includes everything you need to produce a comprehensive beta plan, including:

1. Ready-to-Use Beta Test Plan Template

We’ve developed a beta test plan template that’s complete, easy to work with, and will make you look great. Our template covers more than 20 unique beta planning topics from scheduling, to budgeting, to closing your beta.

2. Detailed Step-by-Step Companion Guide

We’ve included a detailed guide that walks you through every topic found in the template. The guide also includes more than 25 great tips to help build a bulletproof beta.

3. Fully Developed Sample Beta Plan

Lastly, we’ve included a working example of a complete beta test plan using our template. If you’re looking for inspiration as you’re drafting your own beta plan, this is your handy reference.

Download our Free Hardware Beta Test Planning Kit Now

Interested in a plan for software projects? We have one of those, too.

First off, a little clarification is in order. Big Beta isn’t a term we’ve used before, but there’s more to it than just a catchy blog post title. Large consumer and enterprise companies tend to approach beta testing a little differently. They’re more likely to invest in beta as a large-scale program with an entire staff dedicated to its operations. They also tend to have more formal integrations, like single sign-on systems linking their beta management platform (preferably Centercode Connect) to existing customer communities. Thus, to the extent that there’s a “Big Data” future for beta testing, it will most likely start with Big Beta.

What is HP doing that sounded so interesting to us? It’s more complex than they can reasonably describe in a blog post and slide deck, but here’s the gist. HP has invested in researching ways to add computationally-useful meaning and structure to things like social media messages and product reviews. This allows them to feed the underlying meaning of messages into data mining/machine learning algorithms, which are then analyzed as part of a bigger picture.

To borrow from their example, consider a Twitter post like, “I really like this HP 6510 printer, but the paper jams are really frustrating.” HP’s research helps them to automatically infer that this message is positive about the Photosmart 6510 overall but negative about the paper tray. If these inferences are put into some sort of data structure (e.g., assigning +1 and -1 scores in a matrix of pre-determined parameters), they become much easier for a computer to work with. Analysts then have the capability to cross-reference these types of data points against other streams, like sales information and support tickets, to look for correlations and interesting insights. This ultimately provides them with real-time actionable data that they can use to make faster decisions and smarter responses.

So, how does all this relate back to beta testing? Well, for starters, beta testing is a great source for both unstructured and structured data. One of the fascinating things about HP’s research is that they’re now able to merge these unstructured data streams with structured historical data, like customer demographics, data fields from support tickets, etc. to create a much deeper understanding. In beta, unstructured text (feedback reports, daily journals, forum posts) and structured data (bug severity scales, survey responses, user profiles) are constantly being generated together in potentially useful ways.

One possibility would be to use data from beta testing to train supervised learning algorithms that can later be relied upon by the real-time systems. In a beta test, feedback classification is both very important and very easy. It’s trivial to have a beta tester select what aspect of the product a bug report relates to (e.g., the paper tray). Alternatively, beta test managers can work to classify feedback on the back end as part of their feedback management tasks. Either way, classification provides certainty. You can use properly classified data to identify phrases and descriptions customers are more likely to use when describing a paper tray problem, making future predictions more accurate.

Another idea would be to use what’s learned from the real-time systems in the other direction, benefiting the beta program. Insights generated for released products can help beta managers better understand the significance and best course of action for problems and praise encountered during their tests. If certain words, tone, sentiment, etc. correlated strongly with major problems in a released product right before a drop in sales or string of bad reviews, then you now have something you can watch out for algorithmically during beta tests to set off the alarm bells.

What HP’s research shows us is that these types of ideas aren’t pie-in-the-sky anymore. A future where the walls between beta testing and business intelligence are non-existent is very exciting for us, and we’re looking forward to seeing the ways Big Beta uses Centercode Connect to interact with Big Data.

What do you see as the future of beta testing? Please leave a comment!

Want to learn more about Centercode Connect’s integration capabilities?
Click here to request a live demo

Image courtesy of Flickr user torkildr.

Let’s start with some basics:

(1) Incentives are rewards for participation in the beta. You only want to give incentives to the people who earn them. How are they earned? By meeting the expectations you clearly set for your testers at the beginning of your beta. If a tester doesn’t meet his or her responsibilities, then no incentive should be given at the end of testing.

(2) Incentives should remain secret until the test is over. This is one of those lessons companies often learn the hard way. We’d like to save you the trouble. If you announce early on what the incentive will be, there are two common results. If the incentive is amazing, people will start going overboard with their participation (i.e., providing unhelpful fluff) just to make sure they get it. If the incentive isn’t exciting enough, people will quit on your test. Thus, the best practice is to keep the details surrounding your incentive secret until the test is over. In other words, let the tester assume you’ll be fair—and then be fair.

(3) Incentives make a lasting impression on your testers. People don’t usually think about this, but the incentives you give today have a continuing effect on your beta program in the future. By giving an expensive incentive, testers will expect the same in your next test. You might find yourself sifting through excessive, unhelpful feedback and/or disappointing your next batch of testers if you can’t maintain the expensive incentives. Conversely, if you give testers something they don’t value, they’ll feel exploited. If your testers feel like you don’t appreciate them, they’re unlikely to participate next time.

Of course, what you probably want to know is what makes the best incentive. We usually recommend the production version of the product you’re testing. People signed up for a reason, after all. They wanted to use your product. They’d probably like to continue using it. Testers are rarely disappointed when you go this route.

Sometimes, however, it doesn’t make sense to use the test product as the incentive. If your beta product is a $2,000 laptop, testers will understand if the reward for participation is a little less expensive. Another popular incentive is gift cards for major retailers like Amazon. Not only do Amazon gift cards offer great flexibility, they also have the benefit of being instant gratification if you give out redeemable codes. (Bonus: If you’re using Centercode Connect, you can use the Distributed Values feature to pass out gift codes easily and reliably).

Beyond that, you’re free to get creative with your incentives. James McKey, from Symantec, wrote a guest post earlier this year about innovative ways to reward beta participation. You might also consider extra incentives for your best testers. You want to make sure they come back often.

We should also point out that people often appreciate rewards that are more like mementos than incentives (e.g., “Beta Test Team” t-shirts). Reinforcing your beta testers’ special role in shaping the final product is a nice gesture that usually doesn’t increase the cost of your beta tests much.

In parting, keep in mind that the incentive is only one part of the overall impression you leave with testers. Expressing your thanks regularly, engaging with your testers throughout the process, and responding to their feedback is just as important to building good relations. In our experience, incentives aren’t even the primary reason people join beta tests. Often times, beta testers are just fans of your product or technology in general and participate for the exclusivity.

How do you incentivize your beta testers? Please leave a comment!

Want to learn more tips about beta test management?
Click here for our free eBook, 100 Tips for Better Beta Tests.

Image courtesy of Flickr user lemonhalf.

The Details

Q: Will there be an additional fee for access to the forums?

A: Absolutely not. While we think this will be a really valuable resource for our clients, communities like this can be a great selling point for a product, so it will be included with your Connect license at no additional charge. Think of it as an extension of our support.

Q: If I’m not a Centercode customer, can I still access the forums?

A: Unfortunately, no. They’ll be hidden to people who don’t have customer accounts at help.centercode.com. By limiting the forums to customers only, we think we accomplish two things: (1) we can give beta, UAT, and CAT managers a community for talking shop that beta testers don’t mistakenly join; and (2) since everyone will be a Centercode customer, there is a kind of honor code that we believe will facilitate more open communication.

Q: Isn’t that kind of unfair for beta managers who aren’t customers?

A: Not entirely. You still have very viable places where you can discuss beta management topics. For starters, you can ask questions on Quora or answers.OnStartups.com. Both are sites where we regularly participate by answering beta questions. If you’d rather ask a question privately, we have a short form you can fill out at info.centercode.com/question.

We want everyone to have good resources for beta management information, whether or not you’re a customer. The forums are just a special case, which we hope noncustomer beta managers will understand.

Q: Really, though? There’s no way?

A: Well, technically, if you signed up for a 30-Day Free Trial of Centercode Connect, you’d have access to the forums during that time. On the other hand, once you see how much smoother your beta tests can be with Centercode Connect, you might just want to become a paying customer. Consider yourself apprised of the risk.

Q: Okay, I’m convinced. What can I expect to find in the Community Forums?

A: At first, the Community Forums will consist of four sub-forums: (1) Ask the Community, (2) Share Your Experiences, (3) Feature Requests, and (4) Centercode University. If they catch on, we may add more to help focus the conversations. But for now, we think these four will cover the basics.

• Ask the Community is a place where you can ask anything, from simple questions about Centercode Connect to thoughtful discussions about beta management in general.
• Share Your Experiences is aimed at sharing best practices, triumphs, frustrations, and anything else that’s relevant to beta, UAT, and CAT managers (and isn’t better-suited for Ask the Community).
• Feature Requests is just as it sounds. That’s where we’ll be soliciting input on features and functionality that customers would like to see in future versions of Connect.
• Centercode University is a place where we’ll be posting advanced tutorials (articles and videos) that help you get more out of your Connect implementation.

Q: One more question. How do I access them?

A: Customers will be able to view the forums when they use their accounts to log in at help.centercode.com. Also, in Connect 9.4, managers and internal team members will be able to easily access the Community Forums through the Centercode Support bar displayed at the top of the screen.

If you have any questions about the forums, please comment on this blog post. We’ll be happy to fill in more details.

Want to see how Centercode Connect can improve your beta program?
Click here for a Free 30-Day Trial of Centercode Connect.

Image courtesy of Flickr user drain.

Assigned Test Platforms (Available in Connect Enterprise Edition)

Assigned Test Platforms (ATP) is a brand new module within Connect that allows test platforms to be better integrated into beta tests at the project level. ATP will allow testers to link the specific platforms being used in a beta test to the project itself, rather than attaching them to individual pieces of feedback. This will make their test platforms more readily available in views, filters, dynamic tags, and reporting. It also opens up completely new functionality. For example, you can then restrict users from testing with the same test platform in concurrent projects if that might result in version conflicts. And for products that utilize remote provisioning or firmware flashing (e.g., set-top boxes and mobile devices), requesting device-specific information like serial numbers in an ATP makes it possible to integrate with in-house provisioning systems to automate beta build distribution. ATP includes a number of unique features that allow for very flexible implementations and a wide variety of uses. If you’d like to learn more about ATP (beyond the general documentation provided below), please feel free to contact info@centercode.com to schedule a demo.

Distributed Value Enhancements

We’ve greatly extended our Distributed Value system to cover a wider range of scenarios that our clients were facing. Specifically, we’ve added two new methods of distribution (self-selected and  automatically assigned) that each have two variations. This brings the total methods for distributing values to participants to five. They are:

• Automatically Assigned – Values will be assigned sequentially to users who view this resource. This is useful for distributing product keys, incentive codes, and other values that are not preassociated with specific users.
• Self-Selected Dates and Times – Date and time-based values will be presented to users as a list, allowing them to make selections of their choosing. This is useful for scheduling meetings with groups of users. Note that both a date and time are required for this value, and they will be adjusted to the users set time zones.
• Self-Selected – Values will be presented to users as a list, allowing them to make selections of their choosing. This is useful for allowing users to pick a preferred incentive reward.
• Preassigned Dates and Times – Each date and time value will be preassigned to one or more users of your choosing. This is useful for assigning set meeting times to groups of users.
• Preassigned – Each value will be preassigned to the associated user you provide. This is useful for providing assigned values like a shipment tracking number.

Connect Support Bar

Managers or internal teams will notice a new Centercode Support bar at the top of the screen when logged into Connect. When clicked, the Support bar presents quick and easy access to Connect support materials, including our new Community Forums. The ability to Request Help, Report a Bug, Suggest a Feature, visit our Blog, peruse documentation and Release Notes, and discuss beta testing with other Centercode customers is now only two-clicks away. If you’d like to hide the support bar, simply click Centercode Support and it will tuck away until you need it again. This setting will be retained between your login sessions.

Connect Community Forums

The Connect Community Forums are a new initiative, available exclusively for our customers, that creates a place for discussing both beta testing and Centercode Connect. It’s been a long-standing wish among our customers to have a community where they can interact with their peers at other companies, and now they’ll have that opportunity. The Community Forums consist of four subforums:

• Ask the Community is intended for any questions you have about Connect, beta testing, UAT/CAT, or any other relevant topic.
• Share Your Experiences is a place for our customers to share best practices, creative ideas, testing triumphs, and the occasional frustration.
• Feature Requests is dedicated to suggesting new features and functionality for Centercode Connect.
• Centercode University is where we’ll be posting advanced tutorials and innovative ideas to get more out of your Connect implementations and improve your beta tests.

We’ll have another blog post later this week with more information about the Community Forums. Additional details on the release, including Centercode Help articles on the new features and an overview of smaller changes, are available in the full Connect 9.4 Release Notes.

Not using Centercode Connect Yet? Try it for free for 30 days!