At Centercode we believe in technology's ability to make everyone's life better. In the Delta Huddle Podcast, we bring together industry experts and visionaries to share their insights about building products and bringing them to market.
The world of music and audio is wildly creative, and within the art of creating the perfect song or crafting the right sound, feedback is essential. What’s even more creative, and possibly chaotic, is where music, audio, and technology cross paths with each other. It comes with its own set of unique challenges, but also its own realm of unique joys. Especially when it comes to handling a culturally significant musical legacy, or collecting insights from thousands of users across a global audience.
Joining Stefan Stenroos and Chris Rader to explore all these ideas is Jonathan Pardo, Producer at Google-owned VR developer Owlchemy Labs, and formerly of Sonos and Harmonix, the studio behind the Rock Band series.
Throughout his career in both video games and tech, Jonathan has had the opportunity to work on games such as Rock Band VR, The Beatles: Rock Band, and Fantasia: Music Evolved with Harmonix, and recently served as Manager of Beta Quality Engineering at Sonos, helping maintain a Beta Community of over 150k testers.
During our conversation, Jonathan brought some incredible stories to the table, from the importance of developing The Beatles Rock Band with care, to the “chaos mode” of testing, and how QA can be a place where you can find your career and develop yourself further.
In this episode:
- Jonathan's past companies and current projects
- Working on The Beatles Rock Band
- Unique feedback from Yoko Ono, demoing for Tom Hanks and Rita Wilson
- Immersive experiences and the importance of QA
- Sonos, "Chaos Mode", and working with a massive community of testers
- Centercode, templates, and the importance of having structure in testing
- How will AI shape the future of beta testing?
About Our Guest:
Jonathan Pardo has worked in the video game and tech industry for fifteen years. He started his career with a decade at Harmonix Music, developing award-winning titles such as Rock Band VR, The Beatles: Rock Band, and Fantasia: Music Evolved. After Harmonix, Jonathan joined the smart speaker company Sonos, running software beta programs and maintaining the Sonos Beta Community of over 150k testers.
Jonathan is also a passionate VR developer, having founded Trio VR and The Pardo Brothers, the latter of which is actively developing the VR game SAXOPHONE. He is now a Producer at the Google-owned VR developer Owlchemy Labs.
Follow us and listen to the Delta Huddle Podcast on the go:
- Apple Podcasts: https://apple.co/3V5x74Q
- Google Podcasts: https://bit.ly/3Wq0HTT
- Spotify: https://spoti.fi/3PNYCiJ
- Audible: https://adbl.co/3Pvw90V
- Stitcher: http://bit.ly/3JTE46M
- Pandora: http://bit.ly/42vLu7x
Podcast Transcript:
00:00:00
Stefan Stenroos, Technical Trainer, Centercode
Coming up on this episode of the Delta Huddle Podcast.
00:00:05
Jonathan Pardo, Producer, Owlchemy Labs
The one thing I do recall in terms of feedback, in terms of, like, actual tangible feedback about the game, Yoko Ono did come into the office one day and and I remember I was like away from my desk so pissed at missing her walking through the hallways. But, me and a couple of guys actually took it up to I want to say with my brother too, to just, like, walk by the boardroom that Yoko Ono was in, just to, like, get a glimpse of Yoko.
00:00:31
Jonathan Pardo, Producer, Owlchemy Labs
And so and so, you know, we did that. And just like like, like that's Yoko and just get why it's so nervous and anxious about it. Yeah. Yeah. We saw Yoko.
00:00:40
Stefan Stenroos, Technical Trainer, Centercode
Hello, I'm Stefan Stenroos and welcome to the Delta Huddle podcast by Centercode. The world of music and audio is wildly creative, and whether you're trying to craft the perfect song or find the right sound, feedback is essential. What's even more creative and possibly even more chaotic is where music, audio and technology all cross paths with each other.
00:01:04
Stefan Stenroos, Technical Trainer, Centercode
It comes with its own set of unique challenges, but also its own realm of unique joys, especially when it comes to handling a priceless musical legacy with care or collecting insights from thousands of users across a global audience. Now, joining me and Chris Raider to discuss all these ideas is Jonathan Pardo, producer at VR Studio Owlchemy Labs and formerly of Sonos and Harmonic, the studio behind the rock band games.
00:01:36
Stefan Stenroos, Technical Trainer, Centercode
Now, during our conversation, Jonathan had some wonderful stories to share. Everything from handling The Beatles Rock Band with immense care, to the “chaos mode” of testing, and how you can use QA as a stepping stone to find out what you really want to do with your career. Lots and lots of fantastic stories in this episode. As a fellow audio nerd, I had a chance to really dig in deep and nerd out on some really cool topics here, and Jonathan just brings a lot of energy to the podcast, so it was fantastic having him on.
00:02:09
Stefan Stenroos, Technical Trainer, Centercode
Please let us know in the comments what you'd like to see in future episodes of the Delta Huddle Podcast. And if you're listening on Spotify, go ahead and use that question box at the bottom of the screen. This is the Delta Huddle Podcast. And now, Jonathan Pardo.
00:02:30
Stefan Stenroos, Technical Trainer, Centercode
So, Jon, thank you so much for joining us on the podcast today. Wonderful to have you here. Wanted to kick things off by asking, you know, tell us a little bit more about yourself, where you've been and what's coming up for you next at Owlchemy Labs.
00:02:40
Jonathan Pardo, Producer, Owlchemy Labs
Absolutely. Yeah. So I've been in the video game industry, in the tech industry for about 15 years now. So I started in video games at Harmonix Music, so they invented Guitar Hero. When I joined, they were actively working on rock band, too, so I was a huge fan of those games. I'm a musician by trade. I was going to Berklee for music production and engineering, so it was kind of the dream job and I tried doing both, you know, doing a I was on the quality audio assurance testing team, and as I was working on that project and thinking about going back to school because I started in the summer, I was like, I think
00:03:20
Jonathan Pardo, Producer, Owlchemy Labs
If I went back to school, this is the job I would hope to get when I graduated. So what am I doing? Why am I in school? But stay in school? Stay in school, kids. It's good. I got the opportunity through Berklee, so I'm very grateful for Berklee. But yeah, I mean, I basically just dropped out of Berklee after trying to do both at the same time and started my career in the video game industry, which I didn't really anticipate.
00:03:46
Jonathan Pardo, Producer, Owlchemy Labs
Obviously, I was a huge into games when I was a kid and still looked around my N64 no matter where I went. But getting into games in the audio world was really just kind of fortuitous for me. So I was there for ten years doing audio Q&A, also doing product management for one project. Fantasia Music Evolved also was the lead audio composer and sound designer for this mobile game called Beatniks, which was adorable and is unfortunately no longer available.
00:04:18
Jonathan Pardo, Producer, Owlchemy Labs
So don't even look for it. But it was a great, great game with the awesome team, so really got my hands up to try a bunch of different roles, which was really unique as a very creative atmosphere, just artists and musicians and you know, it was it was it was like going to college. It was in some ways even crazier than college as I was doing it my way.
00:04:42
Jonathan Pardo, Producer, Owlchemy Labs
But yeah, I worked on some dream projects, worked on the Beatles rock band. I was a huge fan of The Beatles. And right before that project, my brother, who's also an audio guy, he's an amazing composer, sound designer. He actually was looking for his next thing and I was like, You need to work at this company. So I got him over on the audio team and we got to work directly with each other on the Beatles Rock Band and we grew up I mean, he bought every single album.
00:05:11
Jonathan Pardo, Producer, Owlchemy Labs
He's my big brother, so I just like you, like the Beatles. So I like the Beatles. And yeah, we just loved it. And to be able to dig into those tracks and work on such an amazing, you know, I don't I don't want to call the Beatles an IP, but to work on an and an amazing project with music that we adored and knew like the back of our hands.
00:05:33
Jonathan Pardo, Producer, Owlchemy Labs
So yeah, lots, lots of amazing roles and projects at Harmonix and I'm super grateful for them giving me those opportunities. After that, I started doing my own stuff, so I ended near the end of my time at Harmonix. I started doing a lot more virtual reality. So the lead on Rock Band VR, I was the person on the R&D phase of Dance Central VR, which was another amazing game.
00:06:05
Jonathan Pardo, Producer, Owlchemy Labs
So really kind of just got really excited by that world. And so I started this small development company called Trio VR, and we developed this really kind of artistic but mindfulness based VR app called Circle, and it was another VR app in the music sphere. It was kind of like a meditative music therapy experience. And the idea was you have these three different orbs in front of you and you can engage with them in different ways.
00:06:37
Jonathan Pardo, Producer, Owlchemy Labs
And basically what you're doing is getting enveloped by these orbs, entering a new space and associated with those orbs as music tracks. And so what you're doing is now three more orbs appear in front of you in this kind of amorphous space and you engage with them again. And what you're doing is, you know, unknowingly at first, but eventually you realize as you're building a song and you're building the soundtrack to the experience in a really kind of, you know, low key seated experience, beautiful way.
00:07:04
Jonathan Pardo, Producer, Owlchemy Labs
So lots of fun to work with, work on that project, continue doing more VR stuff. I'm actively working on this title called Saxophone, and it is with my brother. The Parker Brothers is the developer, very creative name and and we are actively speaking to publishers, potential publishers and really excited to be, you know, we have a great demo that we're psyched about.
00:07:30
Jonathan Pardo, Producer, Owlchemy Labs
I was just talking to him actually this morning. We had a quick meeting about it, so it's been fun to do it on the side but yeah, professionally, I worked at Sonos ever after Harmonix. That's where I started working using Centercode. So I was on the beta team there, started as a software beta program manager, eventually started managing the beta quality engineering team, an amazing team of seven engineers across the world that are sitting in centerfield every day except, you know, reviewing the feedback, reproducing these issues, escalating it to the proper teams.
00:08:04
Jonathan Pardo, Producer, Owlchemy Labs
And that was across every project that we were working on hardware, software, integrations. So really exciting work there. And that kind of took my professional, you know, what I was what I was working on in terms of scope significantly grew. I was working on a much smaller teams, much smaller projects, doing a lot of just manual testing, maybe some play tests, i.e. user research tests at Harmonix, but once we got into Sonos using Centercode, it was significantly higher scale, just thousands of testers.
00:08:41
Jonathan Pardo, Producer, Owlchemy Labs
We had a testing group of 150,000 users in there and just like an amazing plethora of opportunities to find unique cohorts of users that matched exactly what we wanted to test. So learned a ton around qualitative data, around quantitative data, how to, you know, make a test plan at that scale. And it really, you know, as they say, you know, the first six months are just like mayhem at a new place.
00:09:10
Jonathan Pardo, Producer, Owlchemy Labs
And I was very properly in a chaos mode trying to get, you know, my feet steady there because Sonos is full of just genius people. And it was really humbling to work on those amazing, amazing products. I was a fan and actually Beta tester before I joined, so I already had a speaker that I got for free and I love the system I have on it every single room and looking at my play five right there.
00:09:38
Jonathan Pardo, Producer, Owlchemy Labs
And so yeah, as an audio guy, obviously it was a dream to work on those projects. And then from there I moved over to my most recent role, which was at Impact Biosystems. So a bit of a left turn. No, no audio focus there. But the reason I was hired is because I have this gaming experience and they were making a mobile app that had animations and 3D models and all of that stuff.
00:10:03
Jonathan Pardo, Producer, Owlchemy Labs
And basically when I was hired for the role was director of content Management or content production, eventually became director of digital product, just managing the app side of development. And we had a great external team that then turned into a great internal team, really smart and talented engineers and animators, 3D modelers, UI designers. I had a personal trainer that was kind of like my right hand person for a long time, and she was, I don't remember a title, but she was athletic manager or something and she was really just talented at understanding.
00:10:43
Jonathan Pardo, Producer, Owlchemy Labs
Basically what we were building was a recovery system for the body. So how do you assess muscle tension? Muscle, muscle, symmetry, those kind of things, assess them and then alleviate them? And so what we were building were two different products. One is the muscle scanner, which you put up to your muscle, and it kind of does this five second little shake on your muscle and it gives you an assessment of how tight your muscle is.
00:11:06
Jonathan Pardo, Producer, Owlchemy Labs
And then the other one was the massager. So just put that up to your muscle and massage it. And we were giving guidance and content for both of those, how to how to assess your muscles and how to alleviates any issues. And so needless to say, I'm far from a athletes. So it was diving into the deep end once again, but learned a ton around that whole world and you know, got super in-depth with like woop and oring and all of the, you know, other health trackers, Fitbit, obviously.
00:11:37
Jonathan Pardo, Producer, Owlchemy Labs
And so, yeah, I learned a ton in that about that world and was working directly with really smart people like Bill Parisi, who's into like fascia and really on the cutting edge of muscle health. One of the coaches from the Dodgers, Brandon something who is a really great guy. So yeah, just like working with people that I had no place working with, but they were more than happy and willing to like give us feedback and their point of view.
00:12:02
Jonathan Pardo, Producer, Owlchemy Labs
They were excited by the product, so it was a really unique opportunity. We shipped the Build the App in October and yeah, I'm really proud of the work we did there. And that brings me to today, or I should say this week. Next week I'm starting my new role at Owlchemy Labs. Oh well, the virtual reality development company owned by Google and they're making amazing work.
00:12:25
Jonathan Pardo, Producer, Owlchemy Labs
I've known this, this team and the founders since the beginning when they made Job Simulator. I guess I wasn't their beginning, but that was the VR beginning for them. And and they've just skyrocketed. They're one of the best in the business. So I'm super pumped to be a producer over there starting in about six days. So it's going to be great.
00:12:45
Stefan Stenroos, Technical Trainer, Centercode
Very nice. Yeah. So you've really covered the gamut of pretty much anything that anyone could imagine from audio to fitness and now into VR, which is awesome. Yeah, I'll engage in some intense flattery here. The Beatles Rock Band was what really got me into the Beatles. I think I sank nice a couple hundred hours into that game. Oh, man, You know, it allowed me to explore the catalog and all that stuff.
00:13:08
Stefan Stenroos, Technical Trainer, Centercode
Yeah. You know, my parents have the CDs and everything, but I think that was kind of the point where, you know, I seriously got into them and started listening to them a lot. So thank you for being a part of that project. And oh, man, it's interesting. You know, you said you didn't want to call the Beatles an IP, right?
00:13:23
Stefan Stenroos, Technical Trainer, Centercode
And I was kind of thinking about this the other day. And even though it's a video game, you know, that video game kind of exists as a part of their catalog of work, in a sense. Yeah, right.
00:13:32
Jonathan Pardo, Producer, Owlchemy Labs
Yes, I think I would agree with that. Yeah. Part of that, right? Yeah. No, totally. I think it's it was you know, I don't know how much they think that is, but like from like a cultural point of view, like we, we took it very seriously. Like the, the intro sequence is, is like a beautiful piece of art just by itself.
00:13:51
Jonathan Pardo, Producer, Owlchemy Labs
It was made by the artist that made all the Gorillaz music videos. So it's very stylized. It's like it basically is one humongous Easter egg, you know, just watching that thing, like if you know, the Beatles and their world and their music videos and their songs. So it's just like every step along the way. Everybody that was working on that project and I can't I can definitely not say this about every project I work on, but that one more than anyone else, it was like we felt like we were handling just like, you know, the Holy Grail or something, and we took it.
00:14:23
Jonathan Pardo, Producer, Owlchemy Labs
It was very daunting. It was a very daunting and so, you know, working directly with the families or the artists from the Beatles like that was it was a lot of fun, a lot of stress. But, you know, a passion project for a lot of people.
00:14:37
Stefan Stenroos, Technical Trainer, Centercode
Yeah. What was that like from a QA perspective? Because, I mean, you have an entire band's history kind of at your fingertips. Yeah. How did you guys make sure that it was handled correctly and everything kind of sounded proper?
00:14:49
Jonathan Pardo, Producer, Owlchemy Labs
Yeah, great question. I mean, that was something we talked about often. So as I said, I was on the audio team, which, you know, I remember when we were working on the project, it was audio team in the pit, as we call it, or we call ourselves AKWA a like a K. So we were in the AKWA pit and it was literally like, if you're in the office, it's like the lowest place you could get to other than the basement.
00:15:14
Jonathan Pardo, Producer, Owlchemy Labs
And so it felt just like this pit. It was just a bunch of like it was a very frat vibe with like a super bizarre group of human beings that all gelled really well together and we were right outside the audio team's doors. And so they actually had rooms with thoughts as luxurious. And they I remember. So my brother was like literally in the door, in the room next to me.
00:15:36
Jonathan Pardo, Producer, Owlchemy Labs
And we would, you know, he would like call me and say like, hey, come check out this track. And he would like be listening to Michelle and it was that song is, you know, a standard if it's anything and it ends with actually a fade out and and he brought me over and he was just like, listen to this.
00:15:57
Jonathan Pardo, Producer, Owlchemy Labs
And he he removed the fade out and just let it play to the end. And George Harrison's playing guitar and he ends it on this like really cheesy major seventh chord. And it's just like you can see you can see them in the in the studio at least. George Martin just being like no, that didn't work. I will fade that out.
00:16:17
Jonathan Pardo, Producer, Owlchemy Labs
So getting this little glimpses into like that world in a way that we've never had before was it was something it was beautiful about. But all of that to say once we were working on it in the game, like we took it super seriously. And so there were some things that we did and rock band that we just kind of we didn't even need to be asked by their team, by the Beatles team to do it.
00:16:43
Jonathan Pardo, Producer, Owlchemy Labs
We just knew the right thing to do. Like we didn't want to drop out audio tracks when you were playing the game and messed the note up, like the idea of us messing with the actual audio of the band. I think we like ducked. You know, if you're playing guitar and you're missing notes in rock band, we ducked the part entirely and we go [static noises]. So we're not going to do that to the Beatles.
00:17:08
Jonathan Pardo, Producer, Owlchemy Labs
That is the Beatles. So we we basically determined like, okay, we were not going to do these at all. We will lower the volume just to acknowledge that you're messing up. But, you know, those are the kind of very micro decisions that made it a really compelling experience. And more than just a game, it was a way to kind of immerse yourself into the music and feel like you're a part of it without it being too gamey.
00:17:35
Jonathan Pardo, Producer, Owlchemy Labs
So yeah, there were tons of little decisions like that around the audio, but we had access to this treasure trove of not only music but banter of the band that when they were in the studio and we used that like we, we actually used that during loading. Do you remember that? Yeah, yeah.
00:17:54
Stefan Stenroos, Technical Trainer, Centercode
I do remember that, yeah. And it was I think it was era specific as well. So if you were playing like Eleanor Rigby, you would get audio outtakes and banter from when they were recording Revolver or Yeah, it was Abbey Road, you know, you get some of that. So that was really, really interesting to me. Just as I mean, I'm a big audio nerd as well.
00:18:13
Stefan Stenroos, Technical Trainer, Centercode
So hearing that is it adds to the experience. So thank you for putting that in. This is another question for the audio nerds out there. Was it tricky trying to get kind of the mono mixes and the stereo mixes to kind of work because I know that there's kind of a certain period where the Beatles were like, Everything's mono, and then they kind of expand it in the stereo.
00:18:31
Jonathan Pardo, Producer, Owlchemy Labs
Totally. Yes. I mean, that was I think that was a an exploration type project, you know? So I know that Caleb Epps who was I don't remember his role on the project, but he was a lead in on many projects. And one of our great audio folks at Harmonix, he and a handful of other people I want to say, I mean, Tsarukyan possibly actually flew out to Abbey Road Studios in London and got the chance to sit and, you know, get these tracks, get these stems.
00:19:00
Jonathan Pardo, Producer, Owlchemy Labs
Worked with Giles Martin, George Martin, son, who is very actively involved with. He's a, you know, esteemed audio engineer himself and producer. And so he had the opportunity or we had the opportunity to work with him on these. And so there was some back and forth. I don't recall the exact process, but of actually, you know, doing the digital work of splitting out the vocal parts and different harmonies, the instruments into different tracks so that we can, you know, do some of our own mixing in the game as necessary for the ability to lower and raise volume.
00:19:38
Jonathan Pardo, Producer, Owlchemy Labs
So. So yeah, it was I mean, you know, all of all of that work was, was so above my pay grade. And, you know, when you are handling the tracks that I had because I had some access to like the the dry vocal stems but they were super compressed like not at all. And I remember there were behind like some lock and key.
00:19:58
Jonathan Pardo, Producer, Owlchemy Labs
It was like some file that was just like if you had access to that, you were lucky and I would get temporary access, you know. So but just to be able to listen to John Lennon's vocals by themselves dry with no effects like those kind of things were, it was really magical to be able to hear hear them in that state, that really raw state.
00:20:20
Jonathan Pardo, Producer, Owlchemy Labs
So, yeah, it was it was a it was a unique, unique thing to have access to those stems.
00:20:25
Stefan Stenroos, Technical Trainer, Centercode
Yeah, absolutely. Sort of bringing this back to testing and feedback and QA: did you receive any kind of really important or like game changing feedback from members of the Beatles family or still living Beatles during development? Because I have to imagine they were pretty closely tied up in how this game was.
00:20:48
Jonathan Pardo, Producer, Owlchemy Labs
Yeah, Yeah, they totally were. Yeah. I don't know the exact story and I know we talked a little bit about this when we first started discussing, but I do know that Dhani Harrison, George Harrison's son, was kind of present at the impetus of the conversation of making this title. And I'm totally probably making this up. I have this memory of hearing maybe from Alex Rigopoulos himself, the co-founder of Harmonix, that they were like out on some vacation in some island, which sounds luxurious, but like Alex, regardless, sitting there with Dhani Harrison having a margarita or something and just started talking about, you know, the Beatles rock band.
00:21:26
Jonathan Pardo, Producer, Owlchemy Labs
And I want to say that was even Dhani's idea. And the other thing that Dhani was really excited about was the idea of actual guitar interface. So instead of just the five buttons, like playing an electric guitar and having, you know, you try to play the chords and the lead parts to different tracks, which we eventually did in Rock Band three, which was a lot of fun and a really unique project.
00:21:49
Jonathan Pardo, Producer, Owlchemy Labs
But but yeah, that conversation started, I want to say, with Dhani Harrison and then, you know, started talking to Apple [Corps] and all them. And the one thing I do recall in terms of feedback, in terms of like actual tangible feedback about the game, Yoko Ono did come into the office one day and, and I remember I was like away from my desk so pissed at missing her walking through the hallways.
00:22:12
Jonathan Pardo, Producer, Owlchemy Labs
But but me and a couple of guys actually took it up to I want to say with my brother to, to just like walk by the board room that Yoko Ono was in just to, like, get a glimpse of Yoko. And so and so, you know, we did that. And just like with like, that's Yoko and just get why it's so nervous and anxious about it.
00:22:31
Chris Rader, VP of Marketing, Centercode
Yeah.
00:22:32
Jonathan Pardo, Producer, Owlchemy Labs
Yeah, yeah. We saw Yoko, but she had artistic feedback. So obviously we're representing the Beatles as these [four] characters. And so she had some direct feedback around Jon Lennon, which was like very interesting feedback and totally legit. Basically, she was like, Those are not John Lennon's eyes, those are not John's eyes. And we were like, I remember the team.
00:22:55
Jonathan Pardo, Producer, Owlchemy Labs
We, the artist in the team in the project were like, What is that what? They're nice eyes. And she's like, they're, they're like something like they're dead. Like their eyes have no feeling. There's no emotion. Those are not Jon's eyes. And so it was either one person or a team of people. They buckled down to fix his eyes and get it right.
00:23:14
Jonathan Pardo, Producer, Owlchemy Labs
And so they analyzed video footage of his eyes and how they move, how they squint when he's smiling, how they look, when he's frowning, how they look. And and and I want to say they did it for everybody because it works. Like you can just look at John Lennon's 3D character. And it's not just this, you know, the strange puppeteer version of him.
00:23:36
Jonathan Pardo, Producer, Owlchemy Labs
It is your feeling like that's John Lennon. And so that's the kind of stuff that you're not just making a video game about the Beatles, like you're making a video games for the Beatles, like with the Beatles and and to be able to have the trust of the families of the Beatles and their inputs and then accepting that input and going back to them and like getting the thumbs up from Yoko Ono that George is are Jon's eyes are good.
00:24:02
Jonathan Pardo, Producer, Owlchemy Labs
Like that's like, you know, it's it's it really makes you feel like you're doing something special. So yeah that they were there were absolutely involved. And I do recall also George Harrison's wife was present at the reveal at E3 and and I got to like even demo it for like it was it was an insane project. I feel like it peaked early.
00:24:25
Jonathan Pardo, Producer, Owlchemy Labs
There was one memory of me, like actually in a luxury suite at a hotel, demoing it to to the Beatles Rock Band, the finished product to Tom Hanks and Rita Wilson, his wife. And I was I found myself literally between them, Tom Hanks playing drums and Rita Wilson singing harmonies with me because it was the first song or first game the way you did vocal, vocal harmonies for.
00:24:56
Jonathan Pardo, Producer, Owlchemy Labs
And and I was, you know, because I'm a vocalist, that was like when I got to work on that. So I knew it really well. So I got this crazy chance to sing next to Rita Wilson, and she was like getting all excited at the end of the song. She's like, What does that mean? She's looking at the results and I'm like, Oh, let's.
00:25:10
Jonathan Pardo, Producer, Owlchemy Labs
It says that we got like fifteen out of twenty vocal harmonies. That's like, insane, like. And she's like, That's good. I was like, Yeah, that's good. And she's like, amazing. Like, give me a big hug. And I'm just like, That's Tom Hanks. This is Rita Wilson. This is strange. So yeah.
00:25:26
Stefan Stenroos, Technical Trainer, Centercode
The question I have to ask is, did you guys play Lovely Rita?
00:25:28
Jonathan Pardo, Producer, Owlchemy Labs
Oh, I don't think so. I don't think so. I don't think it was in the original song list. But yeah, missed opportunity.
00:25:36
Stefan Stenroos, Technical Trainer, Centercode
Missed opportunity. Oh, well, but yeah, we had talked a little bit before recording that. You kind of had that story in your pocket, and I originally thought that it was going to be kind of an audio feedback thing, like, Oh, the sound needs to be tweaked. Instead, it was kind of this personal, almost kind of emotional piece of feedback, like, Oh, John's eyes aren't right.
00:25:55
Stefan Stenroos, Technical Trainer, Centercode
Yeah, it's that's really interesting, right? Because it goes beyond the music. Right. And you really did have to capture their persona. Yes. Very accurately. Right. Because it was beyond simply, you know, they made incredible albums, but they're also kind of this worldwide force. And anyone could pick any of the four of them out in a crowd, Right? Yeah.
00:26:15
Stefan Stenroos, Technical Trainer, Centercode
So that was really, really cool.
00:26:18
Chris Rader, VP of Marketing, Centercode
Just very important. Very important details. Yeah. When you said it earlier, Stefan, like your parents, had the CDs. My parents had the vinyl. So there a little bit of age.
00:26:32
Stefan Stenroos, Technical Trainer, Centercode
Well, well, my, my dad probably had the vinyls too, but he worked for BMG in the nineties. So as soon as CDs came out.
00:26:37
Jonathan Pardo, Producer, Owlchemy Labs
Oh nice.
00:26:38
Stefan Stenroos, Technical Trainer, Centercode
Like, yep, I'm just going to go straight to digital. So yeah.
00:26:40
Jonathan Pardo, Producer, Owlchemy Labs
It's funny on.
00:26:41
Stefan Stenroos, Technical Trainer, Centercode
That, but go ahead.
00:26:41
Chris Rader, VP of Marketing, Centercode
Yeah, definitely better in terms of quality, different quality, different. But what's interesting is that it was a pathway for you, right? So like, yeah, you could have listened to the CDs at the vinyls or whatever it is, but at the time you probably were more into video games. So it was a delivery method to get that, you know, the history and culture and, and something to an audience that is either younger and interested in gaming or they're really passionate about being immersed in something that they're excited about.
00:27:15
Chris Rader, VP of Marketing, Centercode
So when you talk about Tom Hanks and trying to get in there, it's, you’re probably passionate about the Beatles and wanting to get access and see that and being able to play with them, right? That's a whole different level of immersion. I know Dave Perry, when we had our very first episode, he talked about getting the call from Neverland Ranch, about Michael Jackson wanting to play The Matrix, right?
00:27:40
Chris Rader, VP of Marketing, Centercode
So it's like these people. But he probably was really into the Matrix series at the time, right? So there's this connection of, you know, bringing just aspects of that that was more pop culture being the Matrix and whatnot. But that level of quality is so important because it can break the illusion, the immersiveness and the importance of that, right?
00:28:00
Chris Rader, VP of Marketing, Centercode
So when you talk about not wanting to like mess up the guitar, right, like and that was a big part of guitar here, it's an awful if you, if you mess if you miss the couple of a couple of chords like and you fail quickly but you still need that that that cues those cues that the information that we call it like feedback back to the user about hey you you are messing up right?
00:28:26
Chris Rader, VP of Marketing, Centercode
You're not hitting this here's you know try to get them back on course and whatnot. So I love those those level of details while still retaining quality. And you know, the the the the nod to them and saying, yeah, here we're not going to mess up your your legacy. Yeah. I really appreciate.
00:28:44
Jonathan Pardo, Producer, Owlchemy Labs
Yeah. It's that's super That's a really good point. It's it's we didn't take our jobs as QA testers as okay we needed to get to like a reasonable level of quality like this was this was for our own ability to sleep at night. We had to make sure this was at the utmost premium quality and it wasn't just like, yeah, Quality of Quality with a capital Q.
00:29:10
Jonathan Pardo, Producer, Owlchemy Labs
So that was, it was, yeah, it was super unique. And, and I think that everybody, you know, that worked on it, whether you had an interest in the Beatles or not, you just knew that you're working on something special.
00:29:23
Chris Rader, VP of Marketing, Centercode
Yeah, I actually had a similar start to my career like, like you, not the college part of I did not go to Berklee or anything fancy, but I started in QA as well. And it's funny, you talked about the basement. When I worked at WD, we actually had a basement lair and there was that was where like the ultimate quality assurance was going on for the, the actual hard drives and some like clean rooms that people were in, like it looked like, well, like white suits.
00:29:50
Chris Rader, VP of Marketing, Centercode
So like it's basically like hazmat down downstairs. But I was, I was in that the third level of the building where we had quality, but we were still like in our dungeon, which was like a lab. Yeah, yeah. So we would have to go through. Everything was static proof. Everything was like you had to make sure that you went through the proper training.
00:30:08
Chris Rader, VP of Marketing, Centercode
So you're still finalizing electronics and whatnot. But it was, it was really neat to me because like, I worked in the grocery store before that and customer service. So I had some aspect of quality. I was always a tech person. I was really into gaming and building my computers and all that fun stuff, but quality gave me access to people around the company.
00:30:27
Chris Rader, VP of Marketing, Centercode
So like I got to learn what I wanted to do within tech, right? So I was in quality, talk to support people, and I talked to the product management team. I would talk to marketing and the events and I just kind of like built my connections, like, what are what are you people doing over here? I want to I want to learn what you're doing.
00:30:46
Chris Rader, VP of Marketing, Centercode
And you it just takes you on this like cool career, right? Because quality's importance kind of the root of everything being good. But then you can, like, build those connections and like, I ended up in product and now I'm in marketing for whatever reason, but in my nature, someone who cares about quality and someone who cares about.
00:31:07
Jonathan Pardo, Producer, Owlchemy Labs
I completely agree. Yeah. Yeah. I think it's I think it's it's a very good way to get I want to choose my words wisely here because it's really easy to say cue was a good way to get your foot in the door, but that assumes that you know what you want to do and that you're just going to use it as a way to to like, weasel your way to the role that you want.
00:31:38
Jonathan Pardo, Producer, Owlchemy Labs
Like, that's not what I want to say. What I actually want to say is it's a great way to figure out what you want to do because kind of just like going to college undeclared, like you're trying out all these different things, you're figuring out what you like, what you're good at. Being in QA, usually you're in a role where you're a generalist.
00:31:57
Jonathan Pardo, Producer, Owlchemy Labs
And so what you're testing, at least in the video game world, is art, audio, you know, functionality, connectivity, you know. So it's like it's a multitude of different things that you just kind of have to be a generalist at. And what that gives you is if you're embedded, ideally you are you have that opportunity to interface with all of these different disciplines and really get to understand how a game is being built or how an app is being built.
00:32:24
Jonathan Pardo, Producer, Owlchemy Labs
And if you have those relationships, you start to get to figure out like these are the traits that an engineer has. And that's, you know, maybe general generalists, but generally engineers have this trait and that's why they're really good at it. Generally, artists have this trait and that's why they're really good at it, etc. And that's for me.
00:32:42
Jonathan Pardo, Producer, Owlchemy Labs
I was in QA, like I said, for a very long time, like ten years off and on throughout different projects. But like I said, I got my hand at a few different roles and what that did for me was help me figure out what do I like to do and why do I like to do it? And so for people to like, kind of have that opportunity to interface with people like got my hand at product stuff, and I've been doing that off and on for a long time.
00:33:04
Jonathan Pardo, Producer, Owlchemy Labs
I like it for a multitude of reasons and I don't like it for a multitude of reasons, like there's, you know, there's a lot of investment emotionally and creatively that you have to put as a product person. And if you know, I'm a creative person, so I like doing that. But if it's for somebody else's products, you know, I'm working for a company and I'm building what they have asked me to build.
00:33:24
Jonathan Pardo, Producer, Owlchemy Labs
Maybe I'm into it for one project, maybe I'm not for the next. And so that's, you know, there's something emotionally that's difficult for me as a creative person to invest myself into something that I'm not in love with, but I need to be in love with. Like I need to be the one in the room that's in love with this thing.
00:33:40
Jonathan Pardo, Producer, Owlchemy Labs
And what I figured out about myself is I'm a people person. I love to communicate, I love to collaborate, I love process. Because when you're in QA and you see that there's no process, you're not going to hear you're not going to hear squat because nothing's going to get documented.
00:34:00
Jonathan Pardo, Producer, Owlchemy Labs
Nothing's going to get communicated, nothing. There's no transparency. I mean, like likely. And so what I figured out is, you know, a person in this industry that I wanted to do more than anything else is I wanted to make sure that the processes were sensible, functioning and consistent understood across the team. And I was just always trying to influence that.
00:34:24
Jonathan Pardo, Producer, Owlchemy Labs
But that really wasn't my role. And so that's where I figured out project program, production management, all of that stuff is those are the people that get to do that and what they're doing is maybe less creative, maybe less impactful to the actual final product, but they're supporting a team of people to build something. And and if you can be that person, then you're doing is you're bringing cohesion, you're bringing consistency, you're bringing a certain level of lowering anxiety levels because everybody doesn't have to worry about the process.
00:34:56
Jonathan Pardo, Producer, Owlchemy Labs
And so I figured out that that's what I love to do and it's taken 15 years to figure it out, trying a thousand different things and maybe I'll change my mind next year. But but there's, you know, that's the thing about quality is like you're touching all of these different disciplines. You get to try things out. You get to, you know, learn what you think you like and then learn.
00:35:18
Jonathan Pardo, Producer, Owlchemy Labs
Actually, I don't like that. So I think it's a great way for people and they're usually entry level positions. You know, So it's a great way for somebody to kind of figure out, I like this industry. I just don't know what to do, figure out what you want to do.
00:35:33
Stefan Stenroos, Technical Trainer, Centercode
It's the long and winding road. Sorry I had to. But speaking of process and communication and team, so no. Yeah, you kind of kick this off by saying that being at Sonos, at least for like the first six months, was like chaos time, which kind of brought me into this kind of idea. It's like, Oh, when you're in chaos time, that's where some of the most creative and like fruitful things come from.
00:36:00
Stefan Stenroos, Technical Trainer, Centercode
But you also mentioned that you worked with the community of 150,000 testers. So I just love to hear more about what it was like at Sonos, you know, beta testing in that environment. What made that really special for you and what kind of stood out?
00:36:12
Jonathan Pardo, Producer, Owlchemy Labs
Yeah, it was so the product itself from the forefront, it's a great product. So I was really, you know, I didn't feel like there was much that I had to do to like, develop this product, which that was like, you know, if there's a reason that I like working with smaller teams, it was that. But how can you say no to working on this product that is already amazing?
00:36:34
Jonathan Pardo, Producer, Owlchemy Labs
Like you get the opportunity to support even in your the smallest little corner of it, to support this humongous like, amazing premium, high quality product. So I was really excited by that, just that whatever my role was, I was like, If I can support this great. And so going from, like I said, kind of more manual testing, having, you know, managing teams of testers, running very small user research tests to then figuring out how does that how did those test plans relate to these test plans where, like, you know, with every test didn't have a 50,000 users in it, but like they had, you know, tens of thousands of users in it likely.
00:37:17
Jonathan Pardo, Producer, Owlchemy Labs
And so how do we take advantage of those people? That's not bad. But how do I utilize those kind testers to be able to prove out this product, prove success of this feature, you know, confirm quality of some integrations and third party integration, those kind of things. So it really was a project project decision and that was, you know, I had this great manager that you knew at least Chris know Jeremy Barrett, who kind of took me under his wing as a new program manager.
00:37:56
Jonathan Pardo, Producer, Owlchemy Labs
And he, you know, he hired me because of my background. He's like, we need that that point of view on this team. We don't have it. And so that he was excited by that and he just let me, you know, he helped me fill in the gaps in my experience. And what he taught me is like, you don't go into a room, a product manager or whoever is like kind of the key stakeholder and say, What do you want to test?
00:38:20
Jonathan Pardo, Producer, Owlchemy Labs
What do you want to know? Because that the biggest the biggest thing that, you know, people start their sentences with that is a he taught me was the biggest red flag was it would be nice to know dot dot dot because just saying it would be nice to know something that's not a KPI that you can test against.
00:38:42
Jonathan Pardo, Producer, Owlchemy Labs
There's no clear objective of why you're trying to test that. So, so really that was like the goal from that first conversation, like what are we trying to learn and why? And like if we understood those things, it wasn't up to the product manager or the key stakeholder to be able to make the decision around how we're going to test it.
00:39:03
Jonathan Pardo, Producer, Owlchemy Labs
That was our job, But we had to really understand what they were building, how we can test it properly and what were we going to learn and if we learned something that said, you know, this is yes, ready for test, what does that mean tangibly? But also, you know, if it doesn't meet that criteria, what are we going to do about it having a mitigation plan upfront?
00:39:28
Jonathan Pardo, Producer, Owlchemy Labs
These were things that, you know, seem basic to me now, but just like we're really valuable to kind of have a template for life, literally have like my notepad with the five or ten questions that he's like, You should always ask these. And like, I will always ask these. And so, you know, being really rigid at the beginning and then eventually realizing I could find my own way of asking these questions.
00:39:51
Jonathan Pardo, Producer, Owlchemy Labs
So, so for me, it was, you know, taking a step back from the specificity of it, you know, the qualitative verse, quantitative testing, like is this something that we want people to get feedback on? So let's say we're testing airplay. We know that people aren't going to stumble upon airplay. So first of all, we need to announce this feature.
00:40:10
Jonathan Pardo, Producer, Owlchemy Labs
We need to tell them this is something you're actively testing, which if it's a smaller feature that we just want to make sure it doesn't break everything. We might not announce it at all and let people just manually discover it. But if it's something that we knew we wanted a lot of eyes on, you know, okay, let's look at airplay.
00:40:27
Jonathan Pardo, Producer, Owlchemy Labs
It's an iOS only feature. So that's our first core cohort, right? We only want people that have iOS devices. So that was a a key part of the one of the what do they call them tools or whatever. One of those little tiny surveys that they filled out in Centercode to be able to say this is something that, you know, if this is my device that I use for my center or my Sonos system.
00:40:51
Jonathan Pardo, Producer, Owlchemy Labs
So being able to identify. Okay, iOS users. Okay, how about iOS users that have utilized airplay? Like, I don't, I don't recall it. That was specifically something that we had access to, but, you know, kind of diving deep into their data because we did have a lot of access to additional data that we didn't even collect from regular users for our beta testers.
00:41:09
Jonathan Pardo, Producer, Owlchemy Labs
That was something that they agreed to in their terms when they signed up on Centercode. So being able to have access to all of those additional usage data features, we could we could really kind of get a better sense of what users we needed, how to collect them, all of that. And so identifying the user cohort, that's one specific thing, being able to, you know, know how we're going to reach out to them.
00:41:32
Jonathan Pardo, Producer, Owlchemy Labs
So putting the comms together, putting the surveys together, really understanding, you know, making sure not only that they are willing to test, but they're willing to test the specific thing that we're testing and then actually, you know, inviting them to the test, bringing them along for a ride for all of the updates and all of those things that we make of the changes, the bug fixes, and then, you know, reviewing all of that data, which is a lot.
00:41:57
Jonathan Pardo, Producer, Owlchemy Labs
And so that was that team, the better quality engineering team just sitting there day after day reviewing this feedback, working directly with folks, sometimes having to get on the phone with people. It was, you know, their role was kind of this dual quality assurance tester and support person because, you know, at the end of the day, we want to make sure that we don't completely bust people's systems.
00:42:19
Jonathan Pardo, Producer, Owlchemy Labs
We also, if we did bust their system, we don't want to fix it right away. We want to understand why it's busted like, you know. Exactly. Yeah, yeah. It could be this new feature that we had just introduced. It could be just something with their setup. So, you know, we kind of have to, like, ride that line between supporting them and not supporting them too quickly, really getting in our standing of the issue.
00:42:42
Jonathan Pardo, Producer, Owlchemy Labs
So, yeah, it was it was a ton of work for every single project because you kind of had to go back to the drawing board every time. And sometimes, you know, repeated projects would come up or Oh, this is very similar, this is a new music service or engineers introducing that's very similar to the last beta program I ran on music services.
00:43:00
Jonathan Pardo, Producer, Owlchemy Labs
So copy and paste a lot of that. But yeah, you know, you just can't template beta programs as much as you wish you could. There's probably about like 75% of it that you can, but there's always new stuff to test, there's always new surveys to write, there's always new questions to ask. And so really being kind of okay with that, that like every project that I take on, I have to kind of put a fresh pair of eyes on this and and go go after it in a creative way.
00:43:29
Jonathan Pardo, Producer, Owlchemy Labs
That was the fun of it. It kept it really, you know, fresh and interesting.
00:43:34
Chris Rader, VP of Marketing, Centercode
Now get monotonous if you're just running through the same exact mold each time. I think our intersection actually at that point was I came in as you guys were hitting too much of a scale. So I think like, okay, we need to for the scale. And it was like, where can we find that in a 75% template? Where can we I don't want to say cut corners but become more efficient at the way we're managing this specifically, obviously with Centercode and I brought whatever knowledge I could from outside your industry or outside to use your company specific.
00:44:09
Chris Rader, VP of Marketing, Centercode
Yeah, yeah. But it was always I thought that was the interesting thing was the so the importance of process and standardization because it gets you to a point where quicker and then it just takes that little bit extra. The human to apply something to make it, you know, really, really great. Yeah. So there's a lot of processes that can be completely automated and there's a lot of templates to use like that.
00:44:31
Chris Rader, VP of Marketing, Centercode
Maybe it's the standard types of feedback, like we call them, you know, issues, ideas and praise. Like, okay, that's the standard feedback. Now, every team may have specific questions that they want to ask, so they may want a specific survey or your product that has new features that are new technologies that no one's seen. How can you template something that doesn't exist in the world?
00:44:53
Chris Rader, VP of Marketing, Centercode
Yeah, but making room for that, for you to be able to have some space to focus on, that's kind of the key to success. It's like, okay, you need to find the templates, you need to find the standards, you need to find the repeatable processes that need to make those produce this live quality with this much attention, this much effort into it totally.
00:45:13
Chris Rader, VP of Marketing, Centercode
And then you can put all of your skills and abilities and make, you know, get this better information from somebody, whether or not it's a custom survey or responding to feedback. That's one thing that was was kind of cool as we were going through feedback. And this was I think in your Santa Barbara campus with Hassan. Mm hmm. Mm hmm.
00:45:33
Chris Rader, VP of Marketing, Centercode
And we were we were looking at feedback coming through and say, Hey, this is what's taking a lot of time. Mm hmm. We have these standard problems. These are known issues, and we have to respond every single one of them. And is there a way that we can maybe automate this and get them to our support articles? And we came up with these cool like macros and automation to push.
00:45:53
Chris Rader, VP of Marketing, Centercode
That's when I was like something like 20% of the stuff was just starting to get automated. Oh cool. Now you have more time to put into the, the, the nitty gritty stuff that you need to get to get more details.
00:46:05
Jonathan Pardo, Producer, Owlchemy Labs
Totally.
00:46:06
Chris Rader, VP of Marketing, Centercode
So that was, that was my, my fun interaction with also seeing the Sonus campus was for us. Yeah that was yeah that was that was my favorite part of it. But yeah.
00:46:15
Jonathan Pardo, Producer, Owlchemy Labs
Back in the day when offices were a thing, I mean they still are I think but, but last time I visited this, I see this once in a while I went to the beta section. Yeah. And there was one person there, James chan was sitting there alone and he was like, so yeah, yeah. But yeah, I completely agree.
00:46:33
Jonathan Pardo, Producer, Owlchemy Labs
I think that there's like the templates of Centercode like that was, that was something that gave us the ability to like kind of relieve the stress of creating that from scratch, like kind of duplicating those, but not only being able to like just get started quickly, but to be able to compare data for this current project to previous projects like being a we, you know, we did a lot of integrations with Tableau to be able to visualize our data and understand, you know, obviously active projects like how many open issues do we have, you know, what's our SLA is, are we meeting those, those kind of things?
00:47:13
Jonathan Pardo, Producer, Owlchemy Labs
How many high priority issues do we have that we have to address? But, you know, long term to be able to compare against previous projects, How is this project doing compared to previous projects? You know what, people are signing in to the project at all compared to previous ones. Are we engaging with them enough? Are we getting enough feedback?
00:47:31
Jonathan Pardo, Producer, Owlchemy Labs
You know, do we need to start communicating in different ways and new ways and additional ways? And so really kind of, you know, giving us the opportunity to say this project against previous ones, the only way you can really compare that is, is if it is apples to apples. And so having the same types of feedback, the same templates of projects, really not, you know, having too much variability for our sake to be able to look at the data, but also for the user sake so they know what to expect when they're getting into a project.
00:47:59
Jonathan Pardo, Producer, Owlchemy Labs
You know, I've tested before, here's a new project. I got this like the back of my hand and not, you know, completely change anything, especially for our really active users or super users, like giving them the opportunity to have a steady world because we were kind of when we first gotten the Centercode, it was Wild West. Maybe it was like everybody that was a program manager got to create their own unique template and you know, it was a free for all in a way that wasn't really successful.
00:48:28
Jonathan Pardo, Producer, Owlchemy Labs
And so that those templates that we created and eventually, you know, really got consistent. It made our lives easier. It made the lives of the testers easier. And it just, you know, looked better, looked more consistent.
00:48:42
Stefan Stenroos, Technical Trainer, Centercode
That it's awesome that Centercode was able to help you guys standardize all that and visualize all yeah, collect all of it together and just be fantastic. So we were glad to help out there. Unfortunately, we are coming up on time. O brother, this has been an amazing conversation. I think we've covered more bases than I was planning to and you know just really, really cool.
00:49:02
Stefan Stenroos, Technical Trainer, Centercode
Yeah so kind of my last question for everybody I'll start with you, Jon, is we're kind of living in the midst of this kind of a revolution, right? It seems like A.I. is at the forefront, everyone's minds. We have GPT4 out there. We have borrowed, you know, it seems like we're on a new frontier, right?
00:49:19
Stefan Stenroos, Technical Trainer, Centercode
It's like the start of the Internet or Social Media in a way. How do you see A.I. changing QA, Testing beta testing in the future and vice versa. How do you think these two tools are going to interact?
00:49:29
Jonathan Pardo, Producer, Owlchemy Labs
MM Yeah, that's a good question. I mean, it's inevitable AI is going to take over in so many different areas of the tech world. And so I, you know, I can speak from my own experience and from what I've seen and what I'm excited about. I'm no expert with A.I., but in my mind, the human element is always going to be required. I think that there's a lot that can be done around testing to make lives easier. You know, one of the things that I've heard that was like super exciting and scary is some story about somebody that was they they requested captivity to develop some code that was super complex and might have not been chatty, Beta might have just been some A.I., but this A.I. created this code and it was so complex to the point where it was almost incomprehensible by the engineers.
00:51:07
Jonathan Pardo, Producer, Owlchemy Labs
And like, those are the kind of stories that are like, Well, if you can't, like, QA your own work because it's not your own work, that's a worrisome place to be because that you're building something that you don't know what level of security you know it is at, and you have no way of resolving issues if issues do arise.
00:51:25
Jonathan Pardo, Producer, Owlchemy Labs
So there's there's like a really cool usefulness for it to be able to develop code quickly and efficiently. Like awesome, like let's try to figure that out, but also to like have guidelines around what our expectations are. I think that's really important and valuable and processes in place that allow us to do this in a smart way while being effective.
00:51:49
Jonathan Pardo, Producer, Owlchemy Labs
So yeah, I at the end of the day, the people using these products that could be influenced by AI, they're going to want something that they like and want to use. So in terms of testing, people are always going to be necessary. Feedback is always going to be necessary. Like, like you're not going to be able to ask an I.
00:52:09
Jonathan Pardo, Producer, Owlchemy Labs
Do you think this new feature is good? Like that's that's not their place. We are still building things for human beings. So being able, you know, assuming we're talking about consumer products, of course, but and because that's my world, that's where I'm thinking like the human element is always going to be necessary. Being able to have A.I. analyze quantitative data more efficiently.
00:52:30
Jonathan Pardo, Producer, Owlchemy Labs
Like, absolutely. Like, have I built dashboards that were completely looking at the wrong data? And I was telling a product manager, we're good to go. And then I find out, you know, a month later from an engineer that that data point that I was looking at, that data set was the wrong data set, and it was the name of the same thing.
00:52:48
Jonathan Pardo, Producer, Owlchemy Labs
And I'm like, Why is it named the same thing? And they're like, I'm sorry, no one knows that answer. You know, those kind of mistakes that that occur like those are the kind of things that I would love to utilize an AI for to just like, say like, yes, I'm going to manually create this new dashboard. I'm going to find all these data and then I'll hook them together in a way that I hope and pray is right.
00:53:09
Jonathan Pardo, Producer, Owlchemy Labs
I'm going to ask a, you know, data engineer to, like, review it. But there's a little bit of, you know, like like I said, hope and pray. I hope I have the right data being able to utilize AI to actually take a look at. Okay. I want to just ask a question. I want to ask Jes this. You know, I want to be able to say like how many users are actively using this feature and this type of device and they use it for more than an hour, you know, like those kind of questions that would take me hours to build a dashboard for that was the thing that I always hoped for Sonos.
00:53:44
Jonathan Pardo, Producer, Owlchemy Labs
And that was very lofty hopes. Like I don't know if anybody's built something that complex for their product, but that's the kind of stuff that I want access to. And that's not somebody's opinion. That's not. How do you like this feature? Is this meeting your expectations of a feature or your level of quality that you would hope for, level, premium, whatever?
00:54:05
Jonathan Pardo, Producer, Owlchemy Labs
So at the end of the day, qualitative data is going to be human beings. That's always going to be human beings and we're always going to need it and we're always going to need to test it with those people, whether it's one on one user research test or large scale beta test or delta tests, that's that's going to be necessary.
00:54:23
Jonathan Pardo, Producer, Owlchemy Labs
So I'm not too concerned about you guys. I think you're going to have a thriving business for a while. But but there's always opportunities there.
00:54:32
Stefan Stenroos, Technical Trainer, Centercode
Yeah, absolutely. Hopefully we can kind of exist at that crossroads of the human, you know, qualitative and quantitative and kind of merge them together into this kind of perfect imaginary cyborg feedback that just makes products. Yeah. Um, but yeah, that's, this has been excellent. I think we have a lot of really great takeaways from here. So Jon, thank you so much for joining us.
00:55:02
Stefan Stenroos, Technical Trainer, Centercode
So many wonderful stories lately. So glad you could share them with us.
00:55:05
Jonathan Pardo, Producer, Owlchemy Labs
Absolutely. It was my pleasure. Stefan and Chris, always a pleasure seeing you. And I'm so grateful for this podcast and what you guys do. It's an admirable thing to actually listen to people and help people, help product managers, QA people better. People figure out how to ask for people's advice and opinions and what they like and features and stuff like this is a world that I believe in wholeheartedly.
00:55:33
Jonathan Pardo, Producer, Owlchemy Labs
And no matter what my role is, I'm going to be, you know, kind of calling for people to ask, ask what? What somebody wants, ask what somebody likes. Like these are the things that are so basic, that are so easy to miss. So I'm grateful for products like Centercode out there.
00:55:53
Stefan Stenroos, Technical Trainer, Centercode
It's a beautiful. Oh, my pleasure. Thank you so much. Yeah. Thank you for listening to the Delta Huddle podcast. If you enjoyed today's episode, leave us a like or a five star rating. You can also find us on Spotify, Apple Podcasts, YouTube and anywhere else you get your podcasts. We'll see you in the next episode and happy testing.
