As teams introduce Agile and DevOps development practices that emphasize continuous release into their organizations, they’ve taken massive strides to improve the speed and velocity of software development. Like the software they produce, all practices need to keep pace with changes within the industry. And those legacy software testing processes that are unable or unwilling to do so are at risk of being replaced. The same rule applies to beta testing.
Is traditional beta testing at risk?
Beta testing has been around for decades. Today, it remains an essential practice for product development – especially new software development. However, in Centercode’s annual industry survey, we found that while over 7 in 10 professionals use beta testing to collect customer feedback before launch, 6 out of 10 aren’t regularly testing their post-launch releases with customers.
Surprisingly, 9 out of 10 of these same professionals stated that testing with customers is beneficial to development. So why aren’t more organizations taking advantage of one of their most valuable resources after initial release?
Part of the fault lies in a lack of standardized pre-release customer testing processes. The survey showed that only 20% of professionals have mature beta testing processes. Without having a solid pre-launch customer test program, keeping up with update releases is near impossible.
In addition to this, there’s been a significant rise in the adoption of continuous delivery and agile practices (92% of the companies we surveyed use agile or an agile hybrid). This means they are releasing faster every year – and adding more chaos into the mix.
Because most agile teams are using short two-week sprints and continuous delivery is almost keeping up with that rate, trying to conduct a traditional beta test – especially one with reactive processes – seems like fitting a square peg into a round hole.
When we fit the pieces together, we start to see why organizations are struggling to include customer testing and validation in their modern software development practices. As we’ve seen, most professionals are fully aware of the value of customer feedback. But combating the dual challenge of managing reactive feedback collection processes and adapting to shorter release cycles in Agile is becoming the new normal.
What’s causing the friction?
Some organizations are willing to sacrifice quality in the short term to release on time, with or without feedback. After all, they’ll eventually get feedback from customers, right?
The challenge, however, is that the pace of these new development processes means the dev team is working on the next release before they have a chance to get customer feedback from the current one. It could be weeks before issues that arise through post-release feedback channels get validation. It could be months before teams are able to change course. A lot can happen in a single month.
How are customer testing practices evolving to meet modern development needs?
In partnering with some of these industry leaders, research reveals how they’re adapting traditional beta testing to align with faster software delivery. The result is a new testing practice that aligns directly with continuous customer testing called delta testing – “delta” referring to the difference between the live version and what’s in the backlog.
Delta testing is an ongoing Customer Validation program that runs from the initial product launch and continues through the product’s entire lifecycle. It solicits feedback from targeted customers in their live environments, the same as alpha and beta testing has always done. But unlike its pre-launch counterparts, delta testing is continuous, which allows for consistent, customer-driven iterations.
Using this cadenced burst method to collect customer feedback on the latest features and fixes before release creates opportunities to increase overall product quality on a routine basis.
While delta testing is an evolution of beta testing, it does not replace alpha or beta as a method of gathering customer feedback. They are all essential practices with different goals and objectives at distinct points in the product development life cycle.
Aligning Customer Testing with Agile and DevOps
By aligning customer testing with modern Agile and DevOps practices, delta testing brings traditional customer feedback collection methods into the modern age. And it can’t come at a better time. More and more customers expect their products to adapt immediately to their ever-evolving environments. Iterating with continuous Customer Validation is the leg up in quality, speed, and frequency of software delivery necessary to stay two steps ahead of your competition.
Learn more about continuous customer testing and how it enhances Agile and DevOps with the best practices and strategies in this ground level guide, Delta Testing 101.
