At Centercode we believe in technology's ability to make everyone's life better. In the Delta Huddle Podcast, we bring together industry experts and visionaries to share their insights about building products and bringing them to market.
We're excited to announce the next episode of the Delta Huddle podcast! Joining Centercode’s VP of Marketing Chris Rader and Technical Trainer Stefan Stenroos to discuss the best strategies for building a successful beta program is Richard Ball, former Director of Field Testing at Peloton.
This episode is all about building a successful beta program, and more importantly, how to secure buy-in from your stakeholders and the teams around you. There are a lot of challenges to overcome when you’re first establishing a beta program; whether it's starting from scratch with very few resources, or trying to convince other teams that beta tests are worthwhile. It can seem incredibly daunting, but with the right tools and strategies, you can turn otherwise skeptical team members into beta believers.
In this episode:
- The value of having a beta support team and managing resources at scale
- The differences between testing at a large company and being a team of one
- How to successfully pitch a beta testing program
- Recruiting effectively through prioritizing product features
- How product limitations can affect beta tester recruitment
- Overcoming internal objections and the challenge of the "validity check"
- Differences between qualitative results and quantitative sentiments
- What to look for when hiring new beta team members
- Advice for new beta managers starting from scratch
About Our Guest:
Richard Ball is a seasoned professional with over 5 years of experience in the beta industry. He has a proven track record of leading and building successful teams at companies such as Fitbit, Anova Culinary, and Peloton, where he most recently served as the Director of Field Testing.
Follow us and listen to the Delta Huddle Podcast on the go:
- Apple Podcasts: https://apple.co/3V5x74Q
- Google Podcasts: https://bit.ly/3Wq0HTT
- Spotify: https://spoti.fi/3PNYCiJ
- Audible: https://adbl.co/3Pvw90V
Podcast Transcript:
00:00:00
Stefan Stenroos, Technical Trainer at Centercode
This is the Delta Huddle podcast.
00:00:06
Stefan Stenroos, Technical Trainer at Centercode
Welcome to the Delta Huddle Podcast. I'm Stefan Stenroos. Today's episode is all about building a successful beta program and more importantly, how to secure buy in from your stakeholders and the teams around you. There are a lot of challenges to overcome when establishing a beta program, whether it's starting from scratch with a very few resources, or trying to convince other teams that beta tests are worthwhile.
00:00:30
Stefan Stenroos, Technical Trainer at Centercode
It can seem incredibly daunting. With the right tools and strategies, you can turn otherwise skeptical team members into beta believers. Joining me and our VP of Marketing, Chris Rader, to discuss how to make it all happen is Richard Ball. Richard is a seasoned professional with over five years of experience in the beta industry, has a proven track record of leading and building successful teams at companies such as Fitbit, Anova Culinary and Peloton, where he most recently served as a director of field testing.
00:00:59
Stefan Stenroos, Technical Trainer at Centercode
Having Richard on was incredible. He brings a wealth of knowledge and experience, some amazing stories about building teams, overcoming those obstacles, and the candor that he had, his honesty in terms of talking about how to overcome these obstacles and how to push those hurdles down so that you can have the aha moment with other teams where they realize the value of beta testing and the value of having this as a central part of your company.
00:01:27
Stefan Stenroos, Technical Trainer at Centercode
Not only does it improve the quality of products, but as Richard will show you, it also opened up avenues of communication between teams that otherwise may not have been as close to each other. This was an incredibly fun podcast to make. I know I always say that, but this one was really, really fun and I hope you enjoy it.
00:01:48
Stefan Stenroos, Technical Trainer at Centercode
So Richard, how did you originally get your foot in the door when it came to joining this whole kind of beta testing, delta testing universe that we've got going on? Where did you start in user testing?
00:01:57
Richard Ball, Former Director of Field Testing at Peloton
Yeah, I kind of just fell into it, to be honest. That was it. My background before beta was in software quality. Originally I came to the came to San Francisco from the UK in 2009, and I was a software quality tester. My brother had been out here a year and he was working for a software biotech company. And I was kind of in 2009 when everything was kind of not doing so well in the world.
00:02:20
Richard Ball, Former Director of Field Testing at Peloton
And I was a contractor. I had my own consultancy in England and things were drying up really quick. And he said, you know, there’s a job at this company in San Francisco where I work, if you want one? And I was like, Who doesn't want to move to California? I was like, I'm in! Work there for... I think it was about seven years in total and kind of worked my way up the ranks, some kind of software trust engineer to leading the R&D product testing team, which is a lot of fun.
00:02:43
Richard Ball, Former Director of Field Testing at Peloton
And at the end of it I was like, okay, kind of I'm kind of over this now and I kind of want to see what else is out there. I live in San Francisco, next door to Silicon Valley. There's got to be something, not like what I was doing before wasn't fun, but there’s got to be something super interesting.
00:02:55
Richard Ball, Former Director of Field Testing at Peloton
And I've always been a hardware guy. I buy every little gadget that comes out. To my wife's dismay. So I really wanted to do something in hardware. You know, hardware was super interesting for me. Fitness, funnily enough, although I've never been a fitness freak, that whole like data tracking stuff was really interesting and I was kind of looking at the job boards and I saw this role for a field testing program manager at Fitbit.
00:03:18
Richard Ball, Former Director of Field Testing at Peloton
I don't really know what this is, but it looks really interesting. So I'll apply and see what it's all about. It kind of went through the process and they sold me on what they do, not just the products and the success that it was having, but the team behind it, the data that they were getting, you know, the ability to talk to customers way before things are actually being sold was something really interesting.
00:03:41
Richard Ball, Former Director of Field Testing at Peloton
And kind of before I took the role, I had the opportunity to lead the team, which was kind of funny. It was we were talking through the interview process and the hiring manager said to me, as I look, you know, you're good for this role. He's like, But I have another role that I'm thinking of that I'm really interested in putting you for.
00:03:58
Richard Ball, Former Director of Field Testing at Peloton
And that was kind of leading our beta support team, which we were very lucky to have a support team within our beta team, which is really cool. So kind of just fell into it really. And it's the only time I've applied for a job and kind of gone through the traditional way now. Yeah, it's I didn't know it would kind of kick off when it kicked off, and I've kind of just been enamored with it ever since.
00:04:18
Chris Rader, VP of Marketing at Centercode
When you said that it's unusual to have the beta support team, can you elaborate what was the beta team doing and what was the support team doing?
00:04:27
Richard Ball, Former Director of Field Testing at Peloton
Yeah, so we were able to, and something I kind of built out a Fitbit, Peloton as well was typical, I think typical beta programs from what I've kind of known them as, is the program managers. If you want to call the program managers, whatever they're termed as kind of do everything right, they work with the stakeholders, they're looking at the feedback, they're putting it all together.
00:04:46
Richard Ball, Former Director of Field Testing at Peloton
They're doing that, a lot of the presenting, and I think given the size of the community that we had at Fitbit, and the just the sheer excitement that people had at that time for the device that we were doing, we have tens of thousands of people to test and to be able to kind of pull all that feedback in and look at it.
00:05:04
Richard Ball, Former Director of Field Testing at Peloton
We had a it was essentially the way I describe it, is like a customer support team within the beta team. The guys were and the girls were, you know, uniquely responsible for getting the feedback, talking to the customers, talking to the testers, reproducing issues, analyzing everything, looking at the trends, kind of taking the noise and putting it to one side and going.
00:05:26
Richard Ball, Former Director of Field Testing at Peloton
These are the points that are really, really important and work with the program managers to then go, okay, we've got to go back to the stakeholders. So it would allow the program management team to, you know, to go down the program path and to, you know, effectively run their programs at a high level. While the support team was there in the meat of everything, kind of just going through all the feedback as it was coming in, keeping on top of it as best as they possibly could.
00:05:51
Chris Rader, VP of Marketing at Centercode
I almost feel like there's a scale point, right? Like, yeah, you need something like, would you say tens of thousands of people responding to feedback? Yeah, having a layer between someone that's running it all internally and getting all that stuff into the actual like engineering side and the logistics of all that stuff, it makes a lot of sense.
00:06:13
Chris Rader, VP of Marketing at Centercode
And you've been kind of both sides, right? You've seen the smaller scale and an upper scale. Yeah. One interesting question on that was that support team reporting like were each support person reporting up to a program manager and or like you said, it was they reported up to you who led the support team and then you reported to the upper field test program or... ?
00:06:39
Richard Ball, Former Director of Field Testing at Peloton
A little bit. It was more of like direct reporting to myself and then dotted lines to individual program managers, so say we had something working on. I don't know what we were doing at the time since very long time ago. Now, say [Fitbit] Charge2, we had a Charge2 beta program manager who was responsible for that program, and we would then work with that program manager to assign support analyst to that program for a determined amount of time and say, Yeah, you really need some help for the first three weeks of the test because as we know that's when everything comes through and then maybe they can say 50% on that one and they can move 50% into another test.
00:07:06
Richard Ball, Former Director of Field Testing at Peloton
They can kind of spread their time like that and then move across. And it was, yeah, just trying to help them from a support level build out different processes, build out different tools that we could use just to kind of free up the program and just to run the test. Right. To build that test out, run the test bonds, to look at what we were going to test, how we were going to present.
00:07:26
Richard Ball, Former Director of Field Testing at Peloton
Yeah, it's unconventional, but it works. It works well.
00:07:29
Chris Rader, VP of Marketing at Centercode
Yeah, like definitely a scale point. I... it's funny, I don't get to hear it too often. Like, I speak. I understand your the language that you're speaking when you talk about that, like knowing when to pull people on what projects and stuff like that where you're saying like a majority of feedback comes in with within the first couple of weeks there's a nice, you know, hump and then eventually there'll be an update and you'll see another hump.
00:07:51
Chris Rader, VP of Marketing at Centercode
But like managing those resources at scale is really important.
00:07:55
Richard Ball, Former Director of Field Testing at Peloton
And it's unique as well. I mean, luckily at places like Peloton and Fitbit that worked, that's not going to work everywhere, right? We were running tests at a given amount of time, you know, at a given point in time with thousands or hundreds, in terms of what we did at Peloton, people. People with huge amounts of feedback that they wanted to get where it kind of made sense.
00:08:12
Richard Ball, Former Director of Field Testing at Peloton
And if it was more of a I don't know, I don't want to bash any other product, but not all products would get that scale. And it kind of just made sense. Like I worked at a couple of other places by myself on it where it wasn't it didn't make sense to to, you know, have another analyst there to do it all.
00:08:26
Richard Ball, Former Director of Field Testing at Peloton
You could do it all yourself, because that's just the nature of the business.
00:08:29
Stefan Stenroos, Technical Trainer at Centercode
What are the different challenges that you've encountered? I mean, obviously Fitbit sounds like you had a huge team support people around you, different managers, etc. and then you just talked about going to a team of like one where it's like, Hey, I am the guy running this. What are the big differences that you've run into there other than just like, Hey, I have a lot of people to work with or I have a lot of resources to work with.
00:08:54
Richard Ball, Former Director of Field Testing at Peloton
Yeah, I mean, it’s a lot more scrappy, I think, when you're doing it on your own, especially at the beginning, when you're really trying to sell what you're doing. I mean, luckily you've been brought in as an expert or a specialist in that area. So the hard work I think is already done and that they recognize the need for it, which is something fantastic.
00:09:09
Richard Ball, Former Director of Field Testing at Peloton
I think now they're trying to think about this earlier and trying to think what the major differences were, and there actually aren't that many. I think it's just interesting, know these things where. You know,
00:09:18
Richard Ball, Former Director of Field Testing at Peloton
You’re still running the tests in a similar sort of way, but you can just do more. And that's something that we kind of did at Peloton when I came in initially as the only person, I had half a resource that was on loan to me from someone else at the time and we were able to do like a Tread test here and there.
00:09:35
Richard Ball, Former Director of Field Testing at Peloton
But it was very apparent that there were other things coming down the line. And I was talking to my boss at the time and I said, Look, you know, we you know, if we build the team like this, this is what we can give back. You know, we can work on software, we can work on accessories, we can work on second and third source parts for stuff.
00:09:52
Richard Ball, Former Director of Field Testing at Peloton
We can do everything as opposed to, you know, just the one thing here and there. So that that for me was the major difference. But yeah, that was, that was kind of the point that I was, you know, always leading to with that. I think it's just, it's just a scalability thing.
00:10:08
Chris Rader, VP of Marketing at Centercode
And you said that you've at that point of having that team, someone's already made the pitch to have a beta program, so they've kind of already made the investment. There's a budget there, there's resources. Yeah. You've been on that other side, right, where you've had to pitch to have a program like that. Can you, can you talk to that a little bit?
00:10:27
Richard Ball, Former Director of Field Testing at Peloton
Yeah. I mean, it's like going back to what I said.
00:10:30
Richard Ball, Former Director of Field Testing at Peloton
I think it's one thing to have, you know, a person or a small group of people say that, yeah, we need beta at the company. But you're also then I mean, take Peloton for example. We had I was part of the hardware team originally and we had my boss and his boss that was actually Fitbit before. It's very small knit community.
00:10:47
Richard Ball, Former Director of Field Testing at Peloton
And they knew the value of feedback and a beta community and a beta program, right. I then had to come in and sell that to everyone else. Product mechanical, engineering, design, Mark all these people, you know, and those obviously skeptical people going, no, you know, we feel we can get feedback this way and we have this.
00:11:05
Richard Ball, Former Director of Field Testing at Peloton
It's trying to, you know, really work with those, you know, kind of cross-functional stakeholders to sell what you need to build. You know, more importantly, I think at Peloton, I think most important is with our product team, it was they were quick, quick, quick. I mean, everybody knows this Peloton story over the last couple of years, scale, scale, scale, build as much as we possibly can.
00:11:24
Richard Ball, Former Director of Field Testing at Peloton
We've got to satisfy all these needs and trying to then now, I mean, it always comes across as slowing down the process, right? Because they're always like, well, we're going to test you in six weeks. It's going to take time. And just trying to get people on the same page of it's not going to take that much more time If you do it like this, If we can intertwine what we do as part of our product development process from conception of an idea through until post launch, you have, you know, the time built in between all these, you know, different milestones of when we can get your feedback, how we can get you feedback,
00:11:58
Richard Ball, Former Director of Field Testing at Peloton
look at all the different, you know, sectors and people that we can get it from and we can give it to you real time as we move on. And you know, we can check in every week. So you guys are still working in the background while we're doing our testing. So it's not really going to sell that much time and just kind of getting people onto that wavelength of testing isn't there to kind of add a bunch of time.
00:12:17
Richard Ball, Former Director of Field Testing at Peloton
It can be done alongside everything else these days to an extent. I mean, hardware is obviously a tricky one that you have to kind of intertwine that a little bit more intelligently. But it's, you know, it's just trying to get people on that page and understanding, you know, kind of what you do and what you can provide with, you know, the resources that you know.
00:12:34
Stefan Stenroos, Technical Trainer at Centercode
Yeah, it's interesting that you say that recently. We just had Sharon Rylander on our previous episode. She works at Square Panda now, but she's been in user testing since like the dot com bubble. And she told like almost the exact same story. She said there was someone at my company who recognized the value in me and the value in user testing.
00:12:50
Stefan Stenroos, Technical Trainer at Centercode
And then a lot of what I had to do was, you know, almost like a sales pitch going in and saying, like, how do I sell you on this?
00:12:57
Chris Rader, VP of Marketing at Centercode
You want to do a roadshow.
00:13:00
Richard Ball, Former Director of Field Testing at Peloton
I had to that stuff for a year. I called it. We had to go on a roadshow with my boss. I was hugely into getting feedback and she's like, You have to go on a roadshow now to all these other people and sell them on what we're doing and show them what you're going to do and get those early wins.
00:13:15
Richard Ball, Former Director of Field Testing at Peloton
And those are hugely important to kind of just get your head down, you know, run some tests for the first couple of months, get some wins under your belt and then go, this is what we can do. Imagine what we could do if, you know, we had all this other stuff.
00:13:27
Chris Rader, VP of Marketing at Centercode
It’s the same thing. When I worked at Western Digital, we called in an expo. We'd actually go to different areas of the company and we'd bring our products and we'd have people come in to get tested. We had artifacts all over the place of like, Here's bugs that we fixed or things that we've addressed now. And it's like us just kind of shopping around for it.
00:13:48
Chris Rader, VP of Marketing at Centercode
Yeah, we do. We do need, you know, budget and here's what we get, here's how we can help you. We have our like, little sales pitches for what services we can provide. And it's just.
00:13:57
Richard Ball, Former Director of Field Testing at Peloton
Yeah, yeah. We used to do a similar sort of thing every year at Fitbit when I was there at the end of the year, we'd always do like a tester appreciation happy hour, and we would run through, Here's the wins that we've had this year. Here are the bugs that the testers within the company had found and it would really draw people in.
00:14:12
Richard Ball, Former Director of Field Testing at Peloton
And we could we we were sold to the testers as they would get free stuff and be able to give us feedback. So they loved it already, but just kind of showing that the uppers and everybody else that was there and said, look what we're actually getting out of it. We're getting a lot of information that.
00:14:25
Stefan Stenroos, Technical Trainer at Centercode
Absolutely. So, Richard, you've worked with a lot of different products here, Fitbit, Peloton, Anova. Peloton and Fitbit, somewhat similar still in the fitness space, but Anova completely different, culinary tech, etc.. And it sounds like even with products that have been similar, you know, maybe fitness oriented, it's been different challenges and whatnot. What has made each product unique from a testing perspective?
00:14:50
Stefan Stenroos, Technical Trainer at Centercode
You know, what makes each one stand out and how have you had to tackle each one a little differently?
00:14:55
Richard Ball, Former Director of Field Testing at Peloton
This is one of the harder questions that you gave me beforehand. I was trying to work this out well.
00:14:59
Stefan Stenroos, Technical Trainer at Centercode
And I apologize now, I apologize.
00:15:02
Richard Ball, Former Director of Field Testing at Peloton
Raise a lot of good thinking. It made me think about stuff of what's coming and and all that. I think. You know.
00:15:08
Richard Ball, Former Director of Field Testing at Peloton
Fundamentally, I think the markets are different and the users that you work with are different. You know, if you take Anova, for example, and they were you know, we were my primary project, there was not only just the sous vide devices, but we were building the other, you know, the precision seven, which is, which is fantastic. Working with a product like that and working with testers like that, there was a lot of expectations.
00:15:32
Richard Ball, Former Director of Field Testing at Peloton
There was a huge amount of education to give because this is a product that nobody had ever seen before, right? Like it was a lot of like, will it replace my microwave, will it replace my oven. How can it replace my oven? It's it's, you know, built into my house like what is this fit.
00:15:471
Stefan Stenroos, Technical Trainer at Centercode
Yeah.
00:15:47
Richard Ball, Former Director of Field Testing at Peloton
And getting feedback from somebody who is that specific and has very specific expectations was vastly different than somebody you know, on a on an exercise bike that they'd had for X amount of time and had been using and were fully invested in it and had, you know, really lived with it every single day. We sort of saw issues, not issues.
00:16:07
Richard Ball, Former Director of Field Testing at Peloton
What kind of similarities between what I did at Anova and then at that Peloton working on the rower. The rower was new. It was the world's worst secret kept secret at Peloton, everybody knew was coming, you know, a fruit of many years worth of labor, you know, selling people on what it was about, why it could accompany your workout regime, whether it's the bike or the guide or the treadmills matter and kind of trying to understand the market and the testers and what their expectations like and how they would give them to you.
00:16:37
Richard Ball, Former Director of Field Testing at Peloton
It was was very, very different. Yeah. Fundamentally, you know, both cases, you know, we were there to look at issues, identify areas of improvement and customer satisfaction. That remains the crux of what we do. Right. It was and just how to then I guess, siphon that from different types of people was was the biggest difference that I think I noticed.
00:16:57
Richard Ball, Former Director of Field Testing at Peloton
And, you know, having worked in a couple of different places.
00:17:00
Chris Rader, VP of Marketing at Centercode
One thing I've always enjoyed, I've done a couple of food related products in the past. Those those ones always tend to be my favorite. Maybe it's just because of the amount of pictures that you get about things that they're creating. And it was it was always like a results kind of thing. You get to see the results when you have something like a streaming device.
00:17:22
Chris Rader, VP of Marketing at Centercode
The results are they were entertained for 30 minutes or an hour or something like that. You know, with fitness devices, they're healthier, but you don't really get to necessarily quantify so much. But with one with food devices or like grills or ovens or anything that relates that like you get to see a direct result. And I always loved that about testing.
00:17:43
Chris Rader, VP of Marketing at Centercode
I've done something with toaster ovens that was one of my favorite ones. And I, I love telling the story because, like, it's like, Hey, I got to run a test on toast. And everyone's like, What? And I'm like, Yeah, I've done, you know, API integration tests and I've done server tests and security cameras and toast and it's just like, that's always the one that's, you know, eye catching thing, the perception of toast.
00:18:09
Richard Ball, Former Director of Field Testing at Peloton
I mean, the food ones are super interesting because it's. It's hard because the participation side of things is tricky because people don't have an unlimited budget to go and buy food to test with. Right. We had we ran up against the oven where people are like, well, you're going to give me money to go and buy all that stuff that you want me to cook. And yeah, you might get, you know, you know, you'd have to have longer surveys or longer questionnaires to expand like a week or two weeks.
00:18:32
Richard Ball, Former Director of Field Testing at Peloton
And one of the only ones, one of the only, you know, sort of not successful but easier to test over a holiday period. Like we love pitching tests over Thanksgiving and over because people were using it.
00:18:42
Stefan Stenroos, Technical Trainer at Centercode
Yeah.
00:18:43
Richard Ball, Former Director of Field Testing at Peloton
People were going to use it. People are going to cook their sprouts and their turkeys and everything else. Whereas, you know, something like Peloton and a little bit trickier to get participation over the holidays unless there are for all of it.
00:18:53
Richard Ball, Former Director of Field Testing at Peloton
You know, I don't.
00:18:54
Richard Ball, Former Director of Field Testing at Peloton
Want to work out 70 hours. So but it was not that you would work out something that was you know what I mean?
00:19:01
Stefan Stenroos, Technical Trainer at Centercode
Someone's out there doing it. We say that but someone's out there doing it, right?
00:19:06
Richard Ball, Former Director of Field Testing at Peloton
Yeah, There’s some intense people out there, like we would have people on that would... the amount of hours and minutes that they could put into these devices was rivaling our reliability test. It was insane. You know, the passion that these guys had behind, you know what they do, it's fantastic.
00:19:20
Chris Rader, VP of Marketing at Centercode
Yeah. One thing always loved about specific tests is like I'd go talk to engineers or product teams and like, what are we trying to solve or what are you trying to learn? And when they'd come up with questions that related to things that were like more performance and reliability over an extended period of time, narrowing down the audiences that we're looking for that fit that right.
00:19:40
Chris Rader, VP of Marketing at Centercode
So not everyone is the 70 hour a week workout routine. Not everyone's, though. I cook three times a day for six plus people inside my oven. There's people that do that. And sometimes you need to have those people on tests. So hearing what your team, what your stakeholders, what they're trying to learn and then trying to like identify what market within your market, like your little subsegments or personas fit with that is super crucial to making sure that you get the feedback to solve their needs.
00:20:12
Chris Rader, VP of Marketing at Centercode
And they're even happier when they get those results right. I'm sure you guys talked about these people are killing it. They're doing so much more than these other teams. Yeah, and they're not we're not paying them to do all this stuff. It's just they're so happy to be there.
00:20:27
Richard Ball, Former Director of Field Testing at Peloton
Yeah, we would find that it would be really vital sort of early on, like with the rower, for example, that we tested the Peloton, it was largely, I don't say untested because we did a lot of reliability and stuff on it, you know, in the lives of what have you, but using it with real people. We had to look for people that would commit to doing, you know, a good portion of workouts in a week early on.
00:20:46
Richard Ball, Former Director of Field Testing at Peloton
And those are the people we wanted to target early because we wanted to get, you know, them through the machine as much as we could. You can kind of, not phase it out, but kind of make it less and less of a priority as you go through the program. But yeah, there's people out there that will do two and three workouts a day and I struggle to do two or three a week these days.
00:21:02
Richard Ball, Former Director of Field Testing at Peloton
So kudos to that. Thank you for helping You're.
00:21:05
Chris Rader, VP of Marketing at Centercode
You're on vacation right now coming out of vacations.
00:21:09
Stefan Stenroos, Technical Trainer at Centercode
Yeah.
00:21:10
Chris Rader, VP of Marketing at Centercode
Did you use like screeners or how would you narrow that audience like, how would you how would you get those people into a test?
00:21:19
Richard Ball, Former Director of Field Testing at Peloton
Well. We had a lot of those. Sort of criteria that we looked at and not enough to make not a not as much to make it difficult for people to apply. But, you know, for something like a rower at the time we were building, it was we had to keep it totally confidential. And it's a big thing, right? So it's not easy. You can't just get a guy to deliver it and they'll shove it through the mailbox.
00:21:39
Richard Ball, Former Director of Field Testing at Peloton
So it was easier at Fitbit, you know, to kind of go through that stage. We had to look for people that were, you know, specifically within a range of a hub that we were working out. We had a lot we had our main kind of research lab out in New Jersey. So we had to find people within a 20 mile radius of that center so that we could then deliver it to them from that location.
00:22:00
Richard Ball, Former Director of Field Testing at Peloton
Our back end data at Peloton was hugely extensive in terms of workout information. Anything you can possibly track on those machines was logged and it was used throughout the whole company to make things better. We were able to go in and we would run a, you know, a basic screener beforehand to our community and say, you know, are you prepared to work out this much?
00:22:21
Richard Ball, Former Director of Field Testing at Peloton
Are you prepared to do this, that or the other? And we would validate that against what they had been doing, you know, in their normal workouts. So we would find people that say, Yeah, I can for sure do, you know, 8 hours a week of rowing? No problem. I have a bike, I ride all the time and you go and look and go, Yeah, you ride every day. I have full confidence that you're going to do what we do.
00:22:41
Richard Ball, Former Director of Field Testing at Peloton
You know, on the flip side, there were people that said they did tons of workouts there, like you haven't used a bike in six months. And I'm like, Maybe we can put you to off to the side for now. But yeah, you know, we we went through a lot of, you know, screening to satisfy that it was a major priority for our, you know, for our stakeholders that we had a very limited amount of product at the beginning and pretty much all the way through.
00:23:03
Richard Ball, Former Director of Field Testing at Peloton
Right. We started with ten rowers on our first. It's like you got to have people who are going to participate. So we worked very, you know, the program manager on that was amazing. And she worked so diligently to make sure we got the most out of the people that we could possibly get for that, you know, a limited amount of rowers that we had.
00:23:20
Richard Ball, Former Director of Field Testing at Peloton
Um, yeah.
00:23:21
Chris Rader, VP of Marketing at Centercode
It's definitely a balance, right? Like they need to fit your market. They need to fit the market. And on top of that, they need to do be able to participate and engage and give feedback. I'm glad you bring that up. It's something very important to focus on when you're when you're looking at these tests. Yeah.
00:23:41
Richard Ball, Former Director of Field Testing at Peloton
I was on super lucky that we have access to people like that. It's not all hard. Companies will do that very easily. But you know, I talked a lot with oh, we talked a little bit with Alex Larsen at Trimble, you know, and he and the product that he works on was obviously very different, and it was harder to get people to commit to running betas because that was their that's their way of life, right?
00:24:01
Richard Ball, Former Director of Field Testing at Peloton
That's how they get paid the, you know, running the trucking software. But with us, I feel it was a little bit easier. But we still struggled at times to get, you know, to kind of tick all the boxes. And we worked really, really hard with the product team and all of our stakeholders to kind of prioritize what they wanted.
00:24:16
Richard Ball, Former Director of Field Testing at Peloton
Of course they wanted everything possible. Yeah, we had a program manager that worked on that. I was still working on that program, I think was very diligent and very focused on, okay, let's rank everything on what is your very most important thing, and we can then recruit people that way. We'll try and get everything and we'll try and tick all of your boxes for you.
00:24:36
Richard Ball, Former Director of Field Testing at Peloton
But let's look at reality. It doesn't always work that way. So in a perfect world, what do you want to see? So that's kind of how we approached, how we would send, you know, recruitments out to people and then how we would look at it afterwards and go, okay, well, we're ticking all these boxes. You know, similarly with the guide that we were kind of, you know, the guy that was the the hardware device that plugs into the TV and tracked you, we worked with our, you know, in my team a ton on that in terms of human recognition through the camera to be able to identify the movements and what they were doing.
00:25:05
Richard Ball, Former Director of Field Testing at Peloton
And you know, we kind of walked into that eyes wide open in terms of there are so many different variations of people, backgrounds, movements, pictures on the wall. So we had to capture pictures of where people intended to use this device to see what was behind them, you know, figure out what color clothing they were going to wear, because that might clash with that background.
00:25:25
Richard Ball, Former Director of Field Testing at Peloton
And, you know, there was a huge amount of like, you know, huge amount of work went into picking the right people and then, you know, kind of working with them as we went through to make sure that now they're still doing what they should be doing.
00:25:36
Stefan Stenroos, Technical Trainer at Centercode
It's one of the things I really love about a really robust testing program is that it really provokes you to have these open conversations with all the teams within your organization, right? Like you just said, you know, in order to make this camera work in the test, you had to go to AI and machine learning to figure all this stuff out.
00:25:55
Stefan Stenroos, Technical Trainer at Centercode
You know, I think that's one of the things that maybe gets overlooked, you know, a lot of people get into a testing program. They say, okay, we need to check a box or we just need to make sure that it's, you know, fit when it launches. But having that value of like, hey, we're really, you know, promoting cross team communication here is super vital.
00:26:14
Richard Ball, Former Director of Field Testing at Peloton
Yeah, for sure. I mean, whenever you can tell when somebody just wants it to be a tick in the box and I when my kind of my ears perk up, I'm like, no, that's not how this work. If this is what you expect, then this is not we're not going to get along very well. Yeah, but now it's going back to what I said the beginning.
00:26:28
Richard Ball, Former Director of Field Testing at Peloton
It's all about those quick wins. And if you can kind of demonstrate what you can do, it changes people's mentalities and, you know, views fairly quickly.
00:26:36
Chris Rader, VP of Marketing at Centercode
Do you have any do you have any quick wins that you have? Do you have your pocket quick wins for a story that you've told?
00:26:42
Richard Ball, Former Director of Field Testing at Peloton
Well. I don't know if it's quick win, but we had one... Let me think if I if I'm allowed to, I think I can probably tell this now and keep it as vague as I possibly can. I don't want to point fingers that people have.
00:26:53
Richard Ball, Former Director of Field Testing at Peloton
We had a product. That was at Peloton that we were putting out into the public, and it's one that we hadn't tested and it was an accessory. So we heard through the grapevine that this thing was happening. It was being, you know, was going on sale very soon. And I think it was myself and my boss at the time were like, We haven't seen this.
00:27:16
Richard Ball, Former Director of Field Testing at Peloton
We haven't tested this. What's the deal here? I'm like, Oh, it's just it's just this don't worry about it. It's fine. It's been going for a long time. We've tested with a few internal people. No problems, no nothing. And, you know, I think it's okay. We'll be fine. I was like, well, let's let's find some people will test it.
00:27:31
Richard Ball, Former Director of Field Testing at Peloton
You know, we have a resource available here to do this. Like, let's just do this. No big deal. Give us. Yeah, it was. It's for a pair of shoes and give us, you know, 50 or a hundred pairs of shoes, whatever you can spare. We'll get most people able to test it though, with a bunch of people.
00:27:44
Richard Ball, Former Director of Field Testing at Peloton
I think the first iteration we ran, we got some rather concerning feedback from people saying, they don’t sit very well. I don't like how they feel on my feet, you know? So we checked the data. We were very confident in the data that we were receiving with the people that we had, you know, resource to test that there was a problem. We dug into that data just to make sure we were we're 100% right.
00:28:08
Richard Ball, Former Director of Field Testing at Peloton
And then we kind of took it to the, you know, my boss and the uppers and we’re like we have a little bit of a problem here, this is what's happening. And we had to go through many rounds of people questioning, you know, our validity. Right. And so if you're sure about this that you tested these people, maybe they're just a bit complaining or this, that and the other.
00:28:27
Richard Ball, Former Director of Field Testing at Peloton
And it's like we had, you know, the program manager on my team that was running it was amazing to have everything in her back pocket that anybody could possibly ask about. Right. We had everything we we were even on calls with the co-founder and COO. It was like, what are we doing here? We're supposed to be shipping these.
00:28:45
Richard Ball, Former Director of Field Testing at Peloton
And it's like, Well, we've got these problems and it is what they are is fixed. And after that, I think that was that got us on the map pretty quickly. And people were very much like, Oh, this is what you guys can do. And you know, even something as innocuous as a pair of shoes, for example, should be, you know, fairly rigorously tested.
00:29:05
Richard Ball, Former Director of Field Testing at Peloton
I mean, that's a huge part of the experience right? So I try to keep it vague, but obviously it wasn't very vague. It's hard to explain, but yeah, that was kind of a it wasn't a quick win, but it was an early and early one for us that I that's a story I tell a lot because I think it's kind of very indicative of the whole beta realm, right?
00:29:24
Richard Ball, Former Director of Field Testing at Peloton
Where people are like don’t worry about it. We've been through it on a few people's feet pre-pandemic within the office. Everybody loved it. It's like, what's it going to like it? Their employees. And they probably sought you out to test the shoe because that's something they wanted to know. But yeah, once again, and on people, you know, varying width and length and all that other stuff, it was, you know, it was very quick and apparent that there was work that needed to be done and we fixed fixed.
00:29:48
Richard Ball, Former Director of Field Testing at Peloton
But now we did some workarounds and, you know, worked with marketing and, you know, all the other, you know, all the other teams, mechanical engineering and stuff to kind of fix it and and put it out into a market that was largely receptive of it, which was nice.
00:30:03
Chris Rader, VP of Marketing at Centercode
Yeah. I've one thing that you explained that that stood out to me was we were talking with the previous guest, his name is Dzevad. The validity check is always one of my, my favorite things. I've been in beta for a while now. My favorite one is are these the right users? These aren't the right customers. This isn't look like our customers.
00:30:23
Chris Rader, VP of Marketing at Centercode
Our customers wouldn't say that. Right? And you got to go do your due diligence and like they here's who we recruited. This is what they look like and they start poking other holes and trying to find a way to just disprove it. I don't want to have to do more work because I don't believe your data, because it's saying either my baby's ugly or my baby's broken or something.
00:30:41
Chris Rader, VP of Marketing at Centercode
And it's just we were we were just kind of joking around about that. And it's like, that's a thing in our industry. It's like you bring up a problem and you're just kind of using the voice of the customer to say, Hey, dev team or product team, the UX team. This is the feedback coming back from people. We asked you who you wanted on the tests.
00:31:02
Chris Rader, VP of Marketing at Centercode
We got your persona and your segments, and so I know how many people and this is just the validity rundown. I felt that as you said it, like oh no you have been there before.
00:31:12
Richard Ball, Former Director of Field Testing at Peloton
There were even further tests after that where they were like, well, let's try this. What about if we do this? Do you think sometimes you just kind to pick your battles, be like, Okay, yeah, well, that we can do. And then you give us another 50 pair of shoes and we can put them on those people's feet. And you know, well, I'm not, you know, speculating that the response, you know, the results are going to be the same.
00:31:27
Richard Ball, Former Director of Field Testing at Peloton
But I’ve been doing this a while, I feel that there's a it's worthy to at least, you know, going to quote unquote, prove your point. Yeah. And go that route. Luckily, it was Yandara, you guys know Yandara well. Yandara ran that test and she has been doing this for a very, very long time and is the best out there.
00:31:44
Richard Ball, Former Director of Field Testing at Peloton
So she was prepared for all of these conversations where, you know, very high level executives at a company. She had a joined, I think, a mere month before. You know, we we were very prepared for that. You know, you sure this is right or, you know, what about these people that you do this? And she was bang on with everything.
00:31:59
Richard Ball, Former Director of Field Testing at Peloton
So it was it was fun. It was not fun at the time. But it's it's fun to look back on now. Yeah.
00:32:04
Chris Rader, VP of Marketing at Centercode
You guys have the experience. I'll pose the question. You may not have the answer offhand. What advice do you have to someone that's running into that? Someone that is presenting an issue that they found in this test and a product UX engineer is coming at them. What advice do you have, especially talking from a manager? What do you tell your your employee?
00:32:25
Richard Ball, Former Director of Field Testing at Peloton
I'm very, you know, kind of I split my response and to trust the data because data is not everything and what we do right people talk a lot about data these days and how important it is. And don't get me wrong, it's hugely important. But in doing what we do, it's the customer sentiment and the customer experience that is massive.
00:32:42
Richard Ball, Former Director of Field Testing at Peloton
So in addition to providing, you know, here's the data that we got, X percent of people said this was an issue. We also then broke it down into. Here are some comments from people. I think once you start getting comments that have a similar theme to them, which is essentially what we do, right, people start looking at it and going, Oh, I get it now.
00:32:59
Richard Ball, Former Director of Field Testing at Peloton
So once you start hearing people complaining, like it made my foot numb, it made, you know, I made this up and I made that happen and whatever. And then I think because people can kind of understand that experience more than they can numbers, you know, I can anyway, I'm not a numbers guy and I'm not really a data person.
00:33:13
Richard Ball, Former Director of Field Testing at Peloton
I'm not an analyst. I'm very much a sentiment kind of person. And I think, you know, with as long as you can rely upon your data knowing that you have, you know, the right, you know, kind of end value for who you're testing with plus some, you know, sort of customer feeling and experience, I think that goes a long way to help improve your point.
00:33:32
Richard Ball, Former Director of Field Testing at Peloton
And yeah, there will be instances where people don't want to listen to it. They'll push for it and they'll learn the hard way because it happens. And I luckily have not had that happen to me. But I know it happens like, you know, there are products out there that clearly haven't been through testing before like this and stuff that nobody ever told them.
00:33:46
Richard Ball, Former Director of Field Testing at Peloton
It wasn't, well, somebody might have done and it went forward anyway. But yeah, I mean, that's that's kind of the thing that I would tell the team is trust in the data that you're getting and trust in what you're hearing from people. And we know we've got the right set of people. We know we're asking the right questions, trust what they're saying.
00:34:03
Richard Ball, Former Director of Field Testing at Peloton
And then once they once everybody was behind that, I think it made those meetings as awkward as they could have been at times. I remember I was in the car on my way down to vacation in Paso Robles, and I was on the phone with very high level executives, the CTO of the CEO and co-founder. Trying to make the point of this is this is what it is, right?
00:34:24
Richard Ball, Former Director of Field Testing at Peloton
They have the right people. There's little conversations like that that stick in my mind. Yes, At the time it was all good and I had my mother in the back of the car listening to all this stuff. I was like, really good.
00:34:33
Richard Ball, Former Director of Field Testing at Peloton
You know what?
00:34:35
Richard Ball, Former Director of Field Testing at Peloton
I feel confident what we're doing. I just feel less confident with my actual situation of being in a car, I tried to do this with, Yeah, those kind of things sticking your head.
00:34:43
Chris Rader, VP of Marketing at Centercode
Yeah, that's that's definitely good advice. I love the flavor of what you're explaining is using something like a percentage, something very quantitative. This is the number of people. So you're showing impact and following up with more qualitative things and stories, right? The comments that are coming through because it it adds more to the the reliability of the data.
00:35:05
Stefan Stenroos, Technical Trainer at Centercode
It’s the human element, right? It brings in more of like, this is the real user and this is how they feel rather than like this is just a number on a dashboard somewhere.
00:35:12
Richard Ball, Former Director of Field Testing at Peloton
Yeah. And they're getting like it's kind of not specific, but even if there are multiple problems or similar problems, kind of focusing on the top one or two things, I think it's easier for people to digest, right? Like don't go in there saying, Hey, when I ride my bike, it makes a noise. I had the handlebars fall off and the seat goes down.
00:35:29
Richard Ball, Former Director of Field Testing at Peloton
Like, let's kind of focus a little bit on, you know, the most important things so we can tackle that first. I think it's a lot easier for people to digest, you know, smaller bytes of data, right. Than it is just to go like and I have all these problems. What are we going to do? It's like, let's give them recommendations.
00:35:44
Richard Ball, Former Director of Field Testing at Peloton
That was something that Centercode, you know, kind of introduced me to many, many years ago was recommendations right? We're not just here to present data. I know 50% of people thought that was 20% of people about this. It's like give them recommendations by people. People would rather if we you know, if it worked a certain way based on this kind of data, I think people would find really interesting.
00:36:03
Richard Ball, Former Director of Field Testing at Peloton
And there's a lot more digestible at that point too. Whereas, you know, just kind of, you know, punting data over the fence to people on the and here's some problems. 50% of people don't like it. And what are we going to do about it? Like that's not necessarily a beta’s job. But, you know, I think for me, it's very important to at least provide that recommendations say, hey, people are saying this, we think we should do that, that, you know, what do we think about that?
00:36:23
Richard Ball, Former Director of Field Testing at Peloton
I might be an interesting conversation or not.
00:36:27
Chris Rader, VP of Marketing at Centercode
Now, amplifying the voice of the customer is what I was like to say. Like you are the voice. They're not there in the meeting with you. They gave you the data and they're trusting you with the data to go get it.
00:36:39
Richard Ball, Former Director of Field Testing at Peloton
And we did have questions like, can we speak to the users? Like, we're going to answer this?
00:36:45
Chris Rader, VP of Marketing at Centercode
Probably not the way that I want you to, but you know.
00:36:48
Richard Ball, Former Director of Field Testing at Peloton
You're going to ask the your leading questions. But yeah, we've got to look at our leading questions and.
00:36:54
Chris Rader, VP of Marketing at Centercode
And talk about it.
00:36:54
Richard Ball, Former Director of Field Testing at Peloton
Don't want to do that.
00:36:57
Chris Rader, VP of Marketing at Centercode
But there's an art to it. I'm from user research and in my past and there's a way to ask questions that are unbiased and like you're saying, not leading way and that's you're trained to do that with surveys. You're trained to do that with interviews. Yeah, same thing with responding to feedback like you're looking to find the root of the problem.
00:37:16
Chris Rader, VP of Marketing at Centercode
You're not looking to push your, your agenda. Yeah. To shaping the data to something you want to see or don't want to see.
00:37:25
Richard Ball, Former Director of Field Testing at Peloton
Yeah, it's very time consuming.
00:37:27
Richard Ball, Former Director of Field Testing at Peloton
It takes a lot of meetings, a lot of conversations. But I don't have patience in life. But I do with things like this at work and I can, I can get through it relatively easily. If it's something that we believe in and we're behind, like it does take a lot of patience. Patience is a big thing in beta testing, especially when you're trying to establish a program or make a very difficult, you know, point or really, you know, give some very difficult feedback to people.
00:37:49
Richard Ball, Former Director of Field Testing at Peloton
And people want to receive bad feedback in any realm of society, but it's just one of those going out as much as you can.
00:37:57
Stefan Stenroos, Technical Trainer at Centercode
So earlier you were talking about Yandara, who does an amazing job and kind of her value as a great program manager, and you talked about at Fitbit, you had the kind of support team alongside you helping out with all this stuff. So my question is like, what are some of the key things you look for when trying to find new team members that are really going to bring an impact?
00:38:18
Stefan Stenroos, Technical Trainer at Centercode
You just mentioned patience there, so I'm going to assume that's a big one.
00:38:23
Richard Ball, Former Director of Field Testing at Peloton
Luckily at Peloton I was able to build the team from myself and it was half a resource, not in not in ability, but just in time. But who then came onto the team full time, you know, pretty soon after. And, you know, data is surprisingly difficult to hire for now. People might think it's an easy thing to do because it has so much with so many qualities
00:38:47
Richard Ball, Former Director of Field Testing at Peloton
a good program manager, associate manager, analyst, whatever you want to call them, needs to have right. Most importantly for me, I think is passion for the product that we're testing. And then I've had enough experience now where I've had people work on teams that are maybe as passionate as as I think they should be to care enough about what people are saying.
00:39:07
Richard Ball, Former Director of Field Testing at Peloton
It might sound terrible about meant to sound terrible, but I think you have to have an inherent passion for the feedback you're collecting that, you know, we hired a bunch of people that had that passion and had used that device on Peloton devices and and ran and cycled.
00:39:24
Richard Ball, Former Director of Field Testing at Peloton
And yeah, it’s something else that I was very conscious of doing early on at Fitbit, that Peloton is I really wanted to bring in. And it was, you know, it was my first kind of, you know, it's a big hire. Jonathan Ho was the first guy that he was working me and I don't want him to be a friend because I know he'll listen to this and try to was amazing as well.
00:39:44
Richard Ball, Former Director of Field Testing at Peloton
Yandara came, as you guys well know, with a huge amount of beta experience. You know, she'd worked at GoPro, she'd worked at Pax Labs. You know, she she was the best of that and still is, you know, one of the top in the industry by far. I wanted to work with her. I didn't know her. I actually met her at a Centercode conference, the first one down in L.A. I think she we were in a session going through how to implement the JIRA integration at the time, and I just met her randomly and we kind of been connected on LinkedIn.
00:40:16
Richard Ball, Former Director of Field Testing at Peloton
I reached out to her not long after I joined Peloton and I'm like, you know, I don't really know you. So this kind of open I have a role here that I think you'd be superbly interested in. She is an expert in data and she's an expert in program management. And for me, you know, to kind of start with somebody like that and then to help build around them was a no brainer for me.
00:40:36
Richard Ball, Former Director of Field Testing at Peloton
You know, between me, my experience and her experience and bringing in some associate program managers was, you know, something that we could do to help, you know, kind of mentor them, kind bring them up to speed in terms of what we do. People who had never worked in the beta field before, we kind of, you know, mentor and kind of show them what it's all about.
00:40:56
Richard Ball, Former Director of Field Testing at Peloton
I think that was hugely important. Having people from Centercode was obviously very important to talk about for 2 to 3 year. But now, I mean, you know, Travis, for example, taking him from a position of, you know, support to program management was something that he was skeptical of, right? Yeah. He had worked in, you know, customer support for Centercode for many, many years.
00:41:22
Richard Ball, Former Director of Field Testing at Peloton
And that's what he knew. And putting him in front of stakeholders and, you know, directors of product and heads of mechanical engineering was scary. But luckily for both of us, I mean not luckily, he’s a fantastic, fantastic guy, a fantastic resource. He kills it, he owns all of the bike testing about I'm not sure exactly now, but it's been five months and he he killed it.
00:41:46
Richard Ball, Former Director of Field Testing at Peloton
So to have that kind of breadth and I was very lucky. Let's not beat around the bush. I was very lucky at Peloton to have the team that I had, the best team I've ever worked with, probably the best team I will ever work with. We had such an amount of experience. The culture within the team was amazing.
00:42:03
Richard Ball, Former Director of Field Testing at Peloton
We all got along exceedingly well. We still hang out now and do things. I just played mini golf on the on the Quest with with Travis and Jonathan last week. And that was something we did it as part of the team. We would do something like that together. And I think, you know, that binds what everybody was doing and, you know, kind of just exemplifies the relationships that we all had, you know, we were, you know, we're a team, but more importantly, we were friends with each other, which I know people like that you shouldn't really be friends with the people you work with.
00:42:32
Richard Ball, Former Director of Field Testing at Peloton
But I think it's it depends like we made it work and it works fantastically well and we got results and, you know, kind of just structuring the team the way that, you know, you know, and I structure the beginning was indicative of how it went through the, you know, almost three years that I'd worked there, you know, just to find those expert resources and build around them so everybody can learn now helps with, you know, the more junior people that were there maybe two years ago who aren't.
00:42:59
Richard Ball, Former Director of Field Testing at Peloton
So just now, they'll probably attest to that. They couldn’t have done it on the other way. I think it was the best way for them to learn how to how to do what we do well, yeah. Long winded answer, I know, but it's it's something I'm hugely passionate about. And, you know, being laid off from a company is rough, but losing that team was even harder.
00:43:20
Richard Ball, Former Director of Field Testing at Peloton
It's even harder like working with your friends and your best friends, like some of the people that are, you know, I've worked with once or twice before and to not work with them anymore? It’s rough. It’s rough. That's what I miss the most about those guys is not seeing them at our 8am stand up every single day because that was the highlight of my day.
00:43:39
Chris Rader, VP of Marketing at Centercode
Well, fun.
00:43:40
Stefan Stenroos, Technical Trainer at Centercode
Yeah.
00:43:40
Chris Rader, VP of Marketing at Centercode
I think it's nice when when you get people that have experiences, like people that are beta program managers or test managers have the experience. Like you said, they're not always in supply for the most part, pulling from support, pulling from highway or research or anything like that. That's where I tend to see people coming from as they like mature into it.
00:44:06
Chris Rader, VP of Marketing at Centercode
Do you see kind of the same thing or are there any other ones.
00:44:09
Richard Ball, Former Director of Field Testing at Peloton
Really, customer support. I think customer support, they kind of have that relationship and that understanding of how to listen to people and understand what they're really saying. Come to a certain extent. I mean, I worked in QA for a long time, but I I'm not sure it helps, to be honest.
00:44:25
Richard Ball, Former Director of Field Testing at Peloton
Like into the actual testing in the setup of the test actually helps. But, you know, I think the customer focus, I think is the most important thing. And data by far, like anybody can build a process or anybody can build the tool. It's all about that connection you have with your customer and understanding what they're trying to say, because without that, you can have the best process, you can have the best test plan in the world, but without the right people and dealing with them and nurturing them and working with partners, you know, you're out of luck.
00:44:52
Richard Ball, Former Director of Field Testing at Peloton
You're not going to get the result that you want. But yeah, we had a couple of people on the team. I mean, we were actually part of customer support in Fitbit. The field testing team or the beta testing team was part of customer support, which is very uncharacteristic. I think you'll find you'll you'll probably you'll probably find that I guess, you know, knowing all the people that you know, that don't go but typically bought a product or hardware or software or something else, but being part of customer support was it was super interesting, made my job as you know, support manager for the team within made it much easier because I had access on a
00:45:22
Richard Ball, Former Director of Field Testing at Peloton
resources that maybe I wouldn't have had access to. But yeah, it's yeah, find a lot of people come from there and I think a lot of our you know, high performers that hadn't worked in beta came from customer support.
00:45:36
Stefan Stenroos, Technical Trainer at Centercode
Yeah, I like how you almost like I'm like, like The Avengers. It's like I'm putting together a team. You got Iron Man, you got your Hulk over here, Thor over here, all that stuff. Yeah. There's plenty of people out there also who, you know, they might be working run role and their manager goes and it's like, Hey, I want you to run a beta program.
00:45:56
Stefan Stenroos, Technical Trainer at Centercode
And suddenly they're in this world of beta and user testing. Yeah. What's some advice that you'd give to them in terms of building out a program, you know, almost kind of starting from scratch from the ground up?
00:46:08
Richard Ball, Former Director of Field Testing at Peloton
That's my dream. Well, I enjoy doing more than anything else. I think, you know, outside of the customer and them being a pivotal point about, I think getting buy in from all of the different stakeholders is critical. I do that when I work. That's everywhere that I work that when I started, you know, Fitbit included and all that, a lot of it was a place that, you know, Udacity and Anova and Peloton where there was nothing other than, you know, hey a product manager has done some testing on the site, which is what we find quite technical.
00:46:37
Richard Ball, Former Director of Field Testing at Peloton
Like my first few weeks was I'm going to meet as many people as I can and try and understand how the development works of what I'm doing. And I've got to go and make alliances with mechanical engineering, with design, with hardware, with software, try and run through any any qualms they have, you know, kind of any obstacles that they see or any, you know, any uncertainties that they have with what we're doing and try and put that together.
00:47:04
Richard Ball, Former Director of Field Testing at Peloton
I mean, you can't alleviate all of that within your initial meetings with people, because its not possible, to try and understand, you know, where they're coming from. It's kind of like a beta of your beta program, right? We are putting it out there to other stakeholders to go watch your feedback for something like this and kind of understanding what that looks like.
00:47:24
Richard Ball, Former Director of Field Testing at Peloton
I think that's that's the most important thing. I think if you try and run it alone without kind of having the buy in and just forcing feedback on people, I think is really hard for people to take. I think that people find it, you know, very, very time consuming and tricky. But I think once we had built good relationships with, you know, our stakeholders across all the departments, all the companies I worked at, we had those alliances and it was life was much easier and feedback was much more accepted without pushback anymore.
00:47:55
Richard Ball, Former Director of Field Testing at Peloton
I think it was easier for people to take it on board, you know, when you had, you know, it had that alliance with them and they would have an understanding of what you were trying to get out as opposed to by trying to give me feedback for something I know is already good because that's often the case. But that's kind of that's where I really focus my time when I go into a new role and, you know, even on my own, it's as I'm scaling a team or even if I'm not scaling a team, it's just trying to understand how it all fits together.
00:48:18
Richard Ball, Former Director of Field Testing at Peloton
And then working with those, you know, kind of pain points. Work with those people as you go along. So the kind of understand what they want out of what you're doing and how you can help them, I think is is it easier? So for now.
00:48:33
Chris Rader, VP of Marketing at Centercode
I got a good analogy. David Perry was on our first episode. He's a great guy. He had this analogy for hurdles. So he said there's people in the companies that, you know, kind of point out hurdles as a hurdle over there. That's like problems. And what I was picturing, as you were saying it, and you said, you know, you're going to run it alone, it just kind of stood out my head is those hurdles that you're you're going into are those objections.
00:49:00
Chris Rader, VP of Marketing at Centercode
So those each of those stakeholders, those team members from engineering or UX or wherever they are, are going to have their things that they're worried about, things that they're concerned with. And there's their disputes for testing. And what I like is your what you're saying is you get ahead of that, you go knock down those hurdles. Let me let me knock this down for you.
00:49:18
Chris Rader, VP of Marketing at Centercode
And it helps when you have a team you that can now run faster because they don't have to go through the hurdles. A lot of times, if you don't address those hurdles beforehand and you just get testing going, you're going to hit it and you're going to be exhausted each time you have to go over one of those or encounter it.
00:49:35
Chris Rader, VP of Marketing at Centercode
In the middle of testing.
00:49:38
Richard Ball, Former Director of Field Testing at Peloton
So yeah, you’ve got to tell story over and over. And it's just it's so tricky to kind of give the background to what you're doing to people as you need them. You're going to run into, Oh, we have an AI bug, let me go and speak the team. And they're like, What do you guys do? What are you trying to do?
00:49:53
Richard Ball, Former Director of Field Testing at Peloton
You start laughing. It's like, I can't go through this all the time. Like you said, it's exhausting. There's no way you do it. And you can't meet everybody at the beginning. But you can. You can try this.
00:50:02
Chris Rader, VP of Marketing at Centercode
Get the majority of them out of the way and make the path clear.
00:50:08
Richard Ball, Former Director of Field Testing at Peloton
Yeah, I mean, that was my first 60 days, whatever it was. My first probably month at Peloton was just talking to people for half an hour a day. And I know that sounds easy and fun. And it was. But just to get to know people and understand what everybody does and how it all fits together and, you know, getting people on your side and it doesn't always work out that way.
00:50:27
Richard Ball, Former Director of Field Testing at Peloton
But you do what you can.
00:50:30
Stefan Stenroos, Technical Trainer at Centercode
Yeah, Sharon talked about it at Square Panda. She is like the first thing I do after getting here is just have one on ones with everybody. I'll just get to know them and get them on my side so that all this becomes much easier.
00:50:41
Richard Ball, Former Director of Field Testing at Peloton
You'll quickly have a couple of people that you know are going to be trickier than others to work with. And that's fine. You just have to kind of try and understand where they're coming from and what they're doing and and, you know, try and figure out how can I help you? Because everybody can do it and everybody needs help.
00:50:56
Richard Ball, Former Director of Field Testing at Peloton
At some point. There's not a job out there that they can just do and everything is fine. It's there's a there's room for help for everyone. And I know everyone will appreciate it one way or the other.
00:51:07
Stefan Stenroos, Technical Trainer at Centercode
Absolutely. Coming up on time now, but some final thoughts, I'm sure you've heard us say it before, Richard, but we like to use that term tech optimism, like about our belief that technology has that power to make everyone's lives better. Yeah. So I want to ask you, what's something that you're optimistic about in tech right now?
00:51:24
Richard Ball, Former Director of Field Testing at Peloton
I was thinking about this on a ton as well. I was trying to think of a really cool and I haven't been able to think of one a lot of stuff I've been playing around with, I think kind of centers around AI and I know a lot of people are talking about I was trying to go a little bit deeper into points of AI that I find most interesting.
00:51:43
Richard Ball, Former Director of Field Testing at Peloton
I've been playing a ton with ChatGPT recently with the AI powered assistants. Just, you know, as I've gone through each of you is, for example, a different company is trying to, you know, have a have an interview with with an AI powered assistant is super interesting and it's something that I was like, this will never work. But just to, you know, hear all the different pieces of feedback and how it interacts, I think the way that it can be, it can be used to, you know, kind of make life better as opposed.
00:52:08
Richard Ball, Former Director of Field Testing at Peloton
And I know there's a huge conversation around it now in education where, you know, teachers and, you know, education, people are thinking this is a cheating tools opposed to a research tool. It could look at it in different ways depending on how people use it. You know, say, hey, write me an essay about, you know, Abraham Lincoln. It could probably do that for you.
00:52:25
Richard Ball, Former Director of Field Testing at Peloton
Turn it in. All right. But there's a lot of different ways that you can use that tool to get a huge amount of information from. I was also talking to somebody recently about online shopping and selling and how it's going to play a part of that. You know, we're all very used to online shopping, but I was talking to somebody last week and they were saying that, you know, there's companies working on things like think of eBay, for example, or an online marketplace where it's like, I a lot of stuff in my house that I probably don't need anymore.
00:52:55
Richard Ball, Former Director of Field Testing at Peloton
It's very time consuming to go, Hey, I'm going to take this Sonos speaker sorry, and I want to sell it on eBay. They are proactively have to take it. I have to take pictures. I have to research and I have to put it on for a price. And I know there is work going on with people now where you can take your phone and you can have it point at stuff like in your garage and your garage will go or your app will go.
00:53:16
Richard Ball, Former Director of Field Testing at Peloton
You have 5000 dollars worth of stuff in here and it'll rank it by the most demanded stuff on that marketplace platform, right? It's like, Hey, you've got a bike over there. You might think it's worth nothing, but it's actually worth $1,000. You're not using stuff like that. It's super interesting. And what I can get into, I know a lot of people talk about autonomous vehicles.
00:53:36
Richard Ball, Former Director of Field Testing at Peloton
I'm a petrolhead, so I don't not really know electric vehicles at all.
00:53:40
Stefan Stenroos, Technical Trainer at Centercode
Same here. Yeah, same here.
00:53:42
Richard Ball, Former Director of Field Testing at Peloton
You know, the electric car, that's not autonomous. It's great. But I think it's, it's great for efficiency and what have you. But as a petrolhead it's not, it's not for me. But you know, that and robotics as well when it comes to AI powered robotic, it is super interesting and just I think a lot of people are scared of it, which is why I get a lot of this gets a bad rap at the moment because they think it's going to take over and all these other horrible things.
00:54:04
Richard Ball, Former Director of Field Testing at Peloton
But I think when people start to think about how it can like enrich your life and just then mundane, you know, seemingly mundane things you do every day and how it can just, you know, free you up to do so many more, you know, cool things and spend time with loved ones and do all this other stuff. You know, if you can have your phone, sell all your stuff for you so you don't have to do it.
00:54:23
Richard Ball, Former Director of Field Testing at Peloton
It would be it's super interesting to me. I just love reading about all the different ways that it can be used.
00:54:28
Stefan Stenroos, Technical Trainer at Centercode
Like user testing. We just need that aha moment and then suddenly all those hurdles are off to the side. It'll come, I'm sure. Yeah. Well, Richard, this is an awesome conversation. Thank you so much for coming on. I mean, just so much knowledge now. Thanks for having a firehose. That's all.
00:54:44
Richard Ball, Former Director of Field Testing at Peloton
Too much. I'm used to it. I'm English, so we waffle a lot.
00:54:48
Stefan Stenroos, Technical Trainer at Centercode
We appreciate a proper chinwag here.
00:54:56
Stefan Stenroos, Technical Trainer at Centercode
Thank you for listening to the Delta Huddle Podcast. If you enjoy today's episode, follow us on Spotify, Apple Podcasts, Google Podcasts and Audible. And for more about how to leverage Centercode for your next user test, head to Centercode.com
00:55:10
Stefan Stenroos, Technical Trainer at Centercode
We've got plenty of resources, ebooks and more to help you get started. Thank you for listening and we'll see you again soon. Happy testing.
