There are a couple of big differences between traditional beta testing and delta testing. Perhaps one of the most significant is delta testing’s focus on working within Agile environments (vs. waterfall). Many parts of the delta testing methodology are pulled directly from common forms of Agile development. You may or may not have heard the term "ceremony" in regards to your PDLC depending on whether you've been on a Scrum team — in this case, it refers to specific meetings that are required to achieve the best performance. Let's discuss three fundamental ceremonies in delta testing and how they align with Agile methodology.

1. Testing prep meeting (aka sprint planning meeting)

When I was fresh out of boot camp in the Marine Corps, the first official training I received was at the School of Infantry. One of the ideas they teach is the concept of a “line of departure.” The line of departure is an imaginary line that, once crossed, changes the team’s whole mindset. It’s the point where you and your team leave the supporting teams and are officially on the mission. The mission could range from combat to humanitarian. It does not matter. Once you crossed that line you were full speed ahead and operating to achieve your mission’s goal.

The prep meeting is similar to this when starting a delta test. It’s the last stop before you start inviting participants to join your test and send out your product that they’ll be using for the next little while. The goal of the meeting is to ensure (one last time) that everyone is aligned on the goals of your mission — specifically, the number of testers, their demo/technographic profiles, the features being tested, and the date you expect to have initial results (see delta huddle below).

This ceremony really has two similarities to agile Scrum. First, it’s aligned with the sprint planning meeting. You’re not trying to plan out months in advance, you just need to have the information required to get through the next sprint and a general idea of the following one. However, once the work is selected you know what you’ll be doing. Second, it starts a phase similar to a “sprint zero”, or a design sprint. The actual test is not running, but everything is being prepared for it to start.

2. Delta Testing Huddle (aka scrum meeting)

Let’s not get too personal, but have you ever put off checking the status of a project because you were too busy with other projects or demands from your organization? Or, I know we’ve all been in a place where you have valuable information, but no platform to get others to take action on it. Enter the delta huddle.

This might be the most important delta testing ceremony. It establishes a consistent forum for checking in on your project and making changes as your Agile development continues. You’ll want to establish this ceremony at the Prep Meeting, and timebox the meeting to be as short as possible, similar to a standup (try starting with 15 minutes).

What’s happening at these meetings? Three things:

• You review what's happened since the last one (the previous phase)
• You take a look at what’s happening now (current phase)
• You fine-tune what’s going to happen next (future phase).

See some similarities?

Here are the key points for this ceremony…

You’re going to review what happened in the last phase, specifically the features that you focused your testers on. It generally takes a few days for enough feedback to accumulate to make decisions on, so you want to look at the whole phase collectively.

When you’re looking at what’s happening now, the focus is on items that need immediate attention. Are your participants able to use the features you’ve asked them to? Are there any major bugs that are holding up the project? These things will likely stand out to you, but you have to take a look to see them.

Lastly, you need to select what will be focused on during the next phase. If there has been an update to software or firmware you’ll likely want to regress some issues. If there seems to be a problem lurking in a specific feature, perhaps you should ask your participants to focus on it. Don’t neglect what’s happening within your organization. Make sure that the information you’re providing to other teams is being used.

It’s easy to see the parallels between the daily scrum and the delta huddle. Keeping your delta huddle at a specific time on specific days is important, so everyone can get into the routine.

3. Testing Wrap-up Meeting (aka retrospective)

Once the product or release is delivered, everyone moves on to the next one. That makes this third ceremony so easy to skip. That’s why it’s a ceremony — it must happen. Your team needs this time to reflect and improve if you’re going to get better (kaizen).

The whole team should be involved in identifying the most helpful changes that will be implemented in the next round to improve the project’s effectiveness. The wrap-up meeting inspects how the overall project performed. Don’t just focus on what went well or poorly, have an open conversation around little nuances as well. Perhaps something actually did okay, but it seemed to be a little off or did not get the traction you were hoping for. Ask pointed questions that will get direct responses, and don’t settle until you’ve identified the root cause - just like a bug. That said, it’s important to have tact and all conversations should be done with respect for everyone involved.

The importance of knowing your testing stakeholders

A quick note on knowing your audience and your data. The very first wrap-up meeting I went to was attended by myself and the engineer assigned to mentor me into the department. He was not surprised and said that rarely did anyone from the fifteen-person invite list ever show up. If you don’t have data that’s valuable to your audience, each of your ceremonies will end up this way. It took me a couple of months to understand what the stakeholders needed to see in the meetings, and by the time I left people who were not able to attend were asking for the information they’d missed. Yours should be the same way.

This is a quick overview and some best practices around the most important delta testing ceremonies. “Best Practices” are not hard rules, and you might have to modify this a little or a lot for your environment. That’s ok, it’s the principles that matter more than the exact execution. If you give these ceremonies a shot, there’s a great chance you will see positive results quickly.