One of our core commitments is delivering feedback from the number of testers our clients were quoted, giving them a level of predictability that is otherwise unheard of in beta.
Over our more than 10 years successfully running hundreds of beta tests, we managed to achieve an average participation rate of over 90% from our testers. Given that the industry average for private beta tests is typically 20-30% participation, this was a huge accomplishment.
That said, we felt that 90% still sent the wrong message to our clients. Given that we quote a specific number of participants, why should you be happy with anything less? We addressed this by shifting our strategy to achieve engagement from 100% of the quoted testers, resulting in greater predictability for our clients.
Each Managed Beta Test we sell is quoted for a guaranteed number of participants (typically in tiers of 25, 50, or 100) with a set period of weeks. In order to ensure we meet that goal, our beta management team may recruit up to the following tier (e.g. 50 -> 75).
Our team is then responsible for managing the entire expanded tester pool, with the extra testers and any additional feedback given to the client at no cost.
If factors prohibit us from over-recruiting (such as limited units or concerns about limiting exposure), we'll switch to an 80% participation goal — and work hard to meet that goal. For example, a 50-person test would result in participation from at least 40 testers. It's worth noting that we'll still be aiming to achieve 90% engagement or higher, but typically need to hold back a small number of physical units in order to support the test.
We've published a detailed blog post answering this very question. Read it here.
While we're generally able to meet our engagement objectives, the testers are ultimately volunteers, and therefore issues can arise outside of our control. If we're unable to meet our participation objectives, you have two options: