Podcast: Download

Welcome back to Dollars to Donuts. This episode features Leanne Waldal, Senior Director of Product Research at New Relic. We talk about establishing research in an organization for the first time, building up a diverse set of research collaborators, and the pleasure of taking on certain types of challenges.

I’ve seen hopeful examples in startups recently where even though there are only 3-5 people they bring in someone for research and that’s highly unusual. Usually, if you’re starting up a company you have seed money, or you have your first round, the first people you’re hiring are engineers. You’re not making one of your first 10 hires a researcher. I know some example now around the San Francisco Bay Area where they’ve actually brought someone in whose role is to do research really early in the company. And that either points to a certain amount of humility around, oh, just because I am the user it doesn’t mean I know the users. Or, because they’re going into a space they don’t know. – Leanne Waldal

Show Links

• The Secrets of Lyndon Johnson’s Archives, by Robert A. Caro
• The Turnaround
• Leanne on LinkedIn
• Leanne on Twitter
• New Relic
• Autodesk
• Dropbox
• Site Reliability Engineering
• EPIC
• LUMA Institute
• RACI model
• Berkeley School of Information

Follow Dollars to Donuts on Twitter and help other listeners find the podcast by leaving a review on iTunes.

Transcript

Steve Portigal: Well, hi, and if you’re a new listener, welcome to Dollars to Donuts, the podcast where I talk to people who lead user research in their organization. otherwise, welcome back to Dollars to Donuts, the podcast where – okay, you got it.

It’s been a little while since the last episode, and I’m really happy to be back making new episodes for you. In a little bit I’ll talk about how you can support me, and the podcast, but first I wanted to mention an interesting article I’m reading. It’s from The New Yorker magazine, from the January 28, 2019 issue. It’s an article by Robert Caro who is an author of among other things, a multi-volume biography of Lyndon Johnson. This article is adapted from his memoir called Working: Researching Interviewing Writing, and his recollections about those activities are what caught my eye. He describes going to do research at the Johnson Library and Museum and going through boxes and boxes of papers. It’s the hard and tedious work of investigative journalism and what struck me was that amount of inferring, and cross-referencing and delving he was doing – he was coming up with facts and perspective that were not there for the taking but were from analyzing and synthesizing the information that he did have as well as what he didn’t have. Later in the article he talks about the importance of “place” in his research, that to understand Johnson he had to leave New York and really spend time in the Texas Hill Country. He writes “As soon as we moved there, as soon as the people of the Hill Country realized we were there to stay, their attitude toward us softened; they started to talk to me in a different way. I began to hear the details they had not included in the anecdotes they had previous told me.” And finally, he talks about silence in interviewing, citing two characters from fiction – Inspector Maigret and George Smiley, and their tactics to quote keep themselves from talking and let silence do its work unquote, where Maigret fiddles with his pipe and Smiley uses his tie to polish his glasses. Caro has his own brilliant technique: “When I’m waiting for the person I’m interviewing to break a silence by giving me a piece of information I want, I write SU (for Shut Up!) in my notebook. If anyone were ever to look through my notebooks, he would find a lot of SUs.”

Speaking of the connections between journalistic interviewing and user research, I want to recommend the podcast from 2017 “The Turnaround” – it’s interviews with different kinds of journalists about doing interviews, about getting to the story. I learned as much from the contrasts between our different objectives as I did from their best practices that apply directly. Check it out!

Now, obviously, I’m back with more episodes of Dollars to Donuts, I’m taking more of an open-ended approach to new episodes, and they’ll appear as they’re ready, without any specific frequency. It takes a lot to do this podcast and rather than taking advertising, or doing crowdfunding, I want to ask you to support the podcast – and me. You can hire me! I plan and lead user research projects, I coach teams who are working to learn from their customers, and I run training workshops to teach people how to be better at research and analysis. You can buy my books for yourself and your friends and your colleagues – I’ve got two books – the classic Interviewing Users and more recently, a book of stories from other researchers about the kinds of things that happen when they go out into the field – it’s called Doorbells, Danger, and Dead Batteries. You can also give this podcast a star rating and a review on iTunes, and you can review either book on Amazon. With your support, I can keep doing this podcast for you.

Let’s get to my interview with Leanne Waldal. She is the Senior Director of Product Research at New Relic. She has led research at Autodesk and Dropbox as well as running her own research agency for almost 17 years!

Thanks for being on the podcast.

Leanne Waldal: Absolutely

Steve: Let’s start with a basic introduction. Do you want to tell us who you are? What kind of work you do? What your company is?

Leanne: I’m Leanne Waldal. I am Senior Director of Product Research at New Relic. New Relic does application performance monitoring. It usually makes people’s eyes glaze over. Basically, we have a tool that developers and engineers, primarily that group, uses to monitor the traffic on websites and mobile apps. So, if you think about streaming media, online retail, financial services – tons and tons of data and traffic – their site reliability team, or engineering team, might use us to monitor that to make sure nothing goes down. I’m based in San Francisco and I’m a part of the design team which is a part of the product team. The product team is mostly in Portland, Oregon. We have a smaller portion of it in San Francisco and we also have a team in Phoenix and in Barcelona.

Steve: Can we talk a little more about New Relic and who the customers are? You said to make sure that it’s not down, but I’m assuming it’s not just that binary state, like is Amazon up or is Amazon down?

Leanne: It’s to monitor it. So, be watching to see if something happens that’s slower. Usually it doesn’t matter if something is faster. It means you’ve optimized performance and things are running really well. So, if things are going slower or you notice that people are having longer wait times for page loads or checkout loads or media streaming, or whatever sort of thing they’re trying to do in your app. You would have spent a considerable amount of time setting our products up, basically, to then do all that monitoring and raise alerts when something goes wrong.

Steve: Okay. So, it’s not up and down. It’s slow and it could be any part.

Leanne: Exactly. Lag time. Checkout’s not happening in the amount of time that you expect it to.

Steve: Did you use the phrase reliability engineers?

Leanne: Yes. So, the new role that basically Google created about 10ish years ago is the site reliability engineer. They wrote a book about site reliability engineering. So, our target user is usually called a dev ops engineer or a site reliability engineer. If you are a more modern company with the way that you’re keeping track of your website and your apps, then you probably actually have a development operations team, or a site reliability team. If you’re not quite there yet, then these people might be a part of an engineering team, or IT operations or something.

Steve: That’s good context. Then you said, you’re part of the design team which sits in the product team?

Leanne: Yeah. So, at New Relic, engineering and product and design all report up to the chief product officer. Different companies do it different ways. Engineering might be alongside product or design. Design might be alongside. But at our company, if you’re in the product org, you belong to the design team, the product team or the engineering team, or a few different teams that are in the org.

Steve: How long have you been in the role that you’re in?

Leanne: I’ve been at New Relic for six months. I did a similar job at Autodesk, before New Relic, for about a year and a half. And before that I had a similar role at Dropbox for a couple of years.

Steve: How do you compare and contrast those three organizations and what you saw and what the trajectory was in those roles?

Leanne: We could start with Dropbox. Dropbox was private, pre-IPO. It was very much a unicorn. When I started there it had 400-600 people. I don’t remember exactly how many. There was no research. They had a small design team. They had, as a lot of tech companies at that time, being a unicorn, had tons of engineers. And also, because Dropbox is a consumer app as well as a business app, everybody used it, so they felt like they knew who the user was.

Steve: Everybody that worked there?

Leanne: Everybody that worked there, yeah. So, I started setting up a research team there and grew it as the design team was growing. That’s different from Autodesk where I went next. A very old company, very fascinating products with fascinating customers and use cases. And people there who were researchers there, who had been researchers there for a long time. And what I was doing there was combining research and analytics together. At Dropbox, analytics was separate. And I had a small team that was more globally disbursed. At Dropbox, my team was all in San Francisco, to start. Here at New Relic, when I joined, there was one researcher here in San Francisco and one in Portland and a similar sort of tone around a lot of the people who work here at the company. Our site reliability engineers (or have been), so they know a lot about the market and a lot about the users. So, we’re basically here to help the company understand all of the new users it’s acquired since it started because it’s a different company now than it was 10 years ago. Ten years ago, New Relic was a company that was mostly reaching out to developers and engineers to use its product and now we’re mostly focused on enterprise and we have lots of enterprise customers now.

Steve: There’s that classic – you referenced this, right. People think they know they’re customer because they are users.

Leanne: Yeah, we all do.

Steve: But you’re kind of saying yes, to a certain extent. I don’t hear you shutting that down?

Leanne: It’s like yes and the marketplace out there has changed and maybe there’s something new that we’re not looking at right now. And also I’ve noticed that for companies where they maybe used to do contextual inquiry and go out and visit their customers and users and understand them deeply, in the last 10-15 years companies have tended to move to surveys and remote interviews and stepped away from that sort of like deep understanding you get from being in a room with someone, or on a construction site, or on the road with them when they’re using your mobile app, or whatever it is. So, you can see everything around them while they’re using your product or your app. And when you move away from that you lose sight of that holistic story of the customer experience. And then research can come in and do that for you.

Steve: Why do you think there is that movement towards the remote and movement away from the contextual work?

Leanne: Contextual work takes time and effort and can be exhausting. Exhausting to do it and exhausting to come back and know what to do with all that data, how to tell a story out of it and how to decide how it can have impact or value? Or sort of the like, now what do we do? I think also as humans we get really familiar with where we are, and we neglect to notice that the world around us has changed.

Steve: So, that’s back to your point then. We started this company based on work that we’d done and skills that we had and users that we knew, that we were, and the world moves on and shifts and new customers…

Leanne: And it’s not just New Relic or Autodesk or Dropbox. That happens at any company. You can be in banking or in legal. Ask anybody who has been there a long time, compared with a person who just started. They’re having much different experiences of it.

Steve: You described, especially with Dropbox and New Relic, this point at which the company had little or no research, and that is not unique. We know lots of companies like that.

Leanne: Oh yes.

Steve: Do you have a hypothesis about what’s going on? There’s a point at which someone like you starts talking to these companies and starts saying, “hey, here’s what I would do if you had me come in and work with you.” What’s going on beforehand? What’s the point of need or pain that’s identified inside these companies when they realize…

Leanne: Sure. So, what I’ve noticed at the companies I’ve worked at, as well as companies I was interviewing at for this sort of position, and companies I know the story around when they hired someone in sort of like a director or higher role in research, usually something changes in revenue and that change in revenue sets forward sort of like how do we figure out what’s going on. And if they have someone who is currently working there in a role who has had experience in the past with doing market research, or doing churn research, then those people will start to raise the flag of like oh, we have to do research. And then it’s how do we do it. So, that’s one reason that someone brings someone in to do it. Another reason is that there’s sort of like a groundswell of engineers, product managers and designers who are telling the people who make hiring decisions, “we need someone to be in charge of design.” Or, “we need someone to be in charge of research. We’re doing a lot of this ourselves and we don’t feel like we’re doing it well enough.” Or, “we, for example, do lots and lots of interviews. We don’t have time to synthesize the data. So, could you please hire a researcher to work with me.” So, sometimes that happens. Another way it happens, which is how it happened here at New Relic, is that the company decides to hire like a VP or SVP of design and then that person who knows, oh this is what a design team looks like – you have someone who is focusing on content design and language and someone who is focusing on interaction design and visual design. And I also need someone who is focused on research. Sometimes it’s from the top down. Sometimes it’s from the ground up. Sometimes it’s there used to be this role at the company before and someone left or was laid off and then they just didn’t backfill it and then after a while realized oh that’s really a role that’s really important.

Steve: We need that.

Leanne: Yeah, and it’s not – I used to think that maybe it was sort of like a trend, like everybody is starting to be customer centric, or user centric, or whatever that means. So, then they were like, “oh, well Joe down the block is doing that, so we need to do this too.” And what do we do, we hire someone to run research. But, as I have been interviewing for jobs over the year and talking to different companies and doing these jobs, I’ve realized, no, there’s actually something that happens inside the company that causes that change. It’s not so much like copying a competitor or another company that you care about. You might do that with engineering or something, but research is sort of like the first thing to go if you start running out of money. And, I’ve seen hopeful examples in startups recently where even though there are only 3-5 people they bring in someone for research and that’s highly unusual. Usually, if you’re starting up a company you have seed money, or you have your first round, the first people you’re hiring are engineers. You’re not making one of your first 10 hires a researcher. So, I know some example now around the San Francisco Bay Area where they’ve actually brought someone in whose role is to do research really early in the company. And that either points to a certain amount of humility around, oh, just because I am the user it doesn’t mean I know the users. Or, because they’re going into a space they don’t know. They just had a really good idea about it and they really need to understand the space.

Steve: That’s an encouraging sign.

Leanne: Yeah.

Steve: Is there a distinction between – you gave a number of different scenarios where research comes in – do you see a distinction in sort of the context, or the action that’s being taken when bringing in a person to do research vs. bringing in a person to lead/manage/run/build research?

Leanne: Yeah. So, if you’re brought in as a person to do research you’re usually reacting to the things you’re being asked to do. You usually don’t get to pull your head up and do strategic work. You’re doing lots of like compare this design with this, do this survey to answer this question, help me with like interviewing these six users in remote interviews. If you’re being brought in as someone to say like where should research fit into the company, it’s more challenging, but more interesting and exciting because then you’re being told like tell us what we should know? Like where should we go with this? Are you going to focus more on the market research side, or the sales research side? Are you going to focus more on the product design side? What are all the things that we need? What kind of programs do we need? How do we get access to users? How do we interact with users and engage users? There’s just all these little pieces of how a research process and practice runs inside of a company that is great fun to put together because you have to do it based on who you’re working with and the culture of that organization. It’s not just a cookie cutter that you put into a company.

Steve: Is there a trajectory from one to the other? Like someone that’s hired because the need is to do research. Is there a trajectory for that person to start answering the kinds of questions that you’re talking about?

Leanne: Yeah, yeah. I think if that’s sort of like the way you work, or your personality, or the way you’re motivated, you can get there. For example, I have someone on my team right now who I mentioned to the other day like – she’s a researcher and I said, “you could make some day, if you wanted to, a really amazing research operations manager because you’re really good at all these pieces around managing the research and communicating it.” Because half of research is PR and sales. You have to find the people to engage with it, to sell it to at the end. You know you want your research to have impact on product or a marketing campaign or whatever you’re doing. So, you have to gather the people around you. And you have to keep track of, particularly in a B2B company, who are all the people who have to be involved if I want to do visit customers. So, in a consumer company you just go out and find people. You talk to them and you do whatever you want with them. At a B2B company you have to be engaging with customer success, with sales roles – whatever they’re called – account execs, account managers – whatever that company calls them. You have to often also be engaging with product managers, designers, design managers. Engineers, didn’t used to be, just as a stereotype, interested in research and now at some companies they are very interested. And so managing all those types of people that you either have to get approval from, or you need to make sure that they’re bought into what you’re doing, or you need them to come along with you and do it, takes a lot of work. So, there’s this whole new role of research operations that usually helps keep track of that. And I’ve noticed some researchers have a real knack for sort of like managing all those details around their project and then shining a light on it and sort of going out and selling it, sort of like a marketing campaign. They just know who to schmooze with and talk with. And some researchers are really good at doing the research and writing the report and sharing it. I pointed out to her, “this is something you are doing for our team without being asked. Like, that’s a real superpower.”

Steve: Does that start to change the – I think you said this, at least indirectly – does someone who is shining around operations, like your colleague here, does that start to change how research is perceived or experienced by others in the organization?

Leanne: Yeah. Especially if you’re an organization that’s not used to research. Or they’re used to doing it themselves. So, they’re not used to partnering with someone else and they’re not used to having someone say like, “oh that’s not really a survey. That’s more of an interview.” Or, “oh, that’s not really an interview. We need to actually go watch people to do this.” Or, “this can’t be answered in an interview survey, or watching people. We need them to be in a diary study.” They aren’t used to having someone bring in all these different methods. And they aren’t used to having someone sort of like find some insights and then tell them. So, that’s why it’s so important to have a collaboration. I want you with me while I’m doing this research so that when I tell you the pens need to be blue, not red, you were there to hear everyone say they like blue better than red, instead of throwing the report over the wall.

Steve: I hadn’t really thought before of operations as a – I’m not sure you’re saying this – but, almost a Trojan Horse. That there are these sort of tactical objective problems to be solved under operations. How are we going to find these people? What is it going to take logistically? What is it going to take legally and so on? That starts to change some of the conversations maybe that can happen internally. Like you said, “this is your problem. We recommend you go about it this way.”

Leanne: Yeah.

Steve: I hadn’t thought of that under an operations lens myself.

Leanne: It’s really more in a B2B company than a consumer company because you have to manage so many relationships and oftentimes a researcher wants to focus on sort of like the details of the research. Like ask any researcher, they don’t want to do all the details around recruiting and scheduling people – or, most don’t.

Steve: Right.

Leanne: And so, you want someone who can do that. And then sometimes researchers also aren’t the people who know how to best sell their work or edit it down into like the three bullet points for an executive presentation. And it’s helpful to have somebody who knows how to do that. You know just sort of like editing and presenting and PR and marketing. And also knowing who to market it to? Like, oh, I know that there’s this sales VP in this office in Toronto who is going to be interested in this. That mentality of like keeping all these dots rolling together. And if you have someone who knows how to do that and also understands how to do research and what goes into research, that’s what I see makes a really good manager from an operations perspective. There’s the people management side too. Being able to mentor and coach and take care of people. But there’s more and more in B2B research a need for someone specifically in this operations role to just sort of like help everybody and raise the brand of the team and the company, or in the product team or the design team.

Steve: It’s so cool that we’re at a point in this field and this practice where we acknowledge, as we always did, that there’s a number of very different kinds of skills and expertises that are required. And you just outlined a whole bunch of them. But that we’re able to sort of say, and not be laughed out of the room, like oh, that might be different people. You talked about collaboration and collaboration with people with different strengths or superpowers.

Leanne: Yeah. And another expertise and skill that’s being introduced more into companies is the academic anthropologist who is now working inside companies on a research team. Like EPIC used to be primarily all academics and now there’s a bunch of people from business involved in EPIC, and also people who used to only be academic anthropologists and ethnographers working on research teams, inside tech companies or consulting agencies, or whatever. So, I recently hired a few anthropologists on my team. One who is a former professor, one who had applied anthropological research practices for market research. Another who had done research with an NGO, but also worked inside of a company recently. And I think that when you have that academic background and you bring it into a company, it’s like a special sauce. You also bring in a special sauce if you’ve been working in tech for 15 years and you know the ins and outs of how a product team works, but I like mixing those two things together because people learn off of each other.

Steve: You like to mix the special sauce.

Leanne: Yeah.

Steve: So, it’s an extra special sauce.

Leanne: Yeah. So, for example, we decided as a team that we wanted to look at all of the internal blog posts and other things that had been written about customers and users, from the point of view of people inside the company, because my team is new to this company. We were like okay, let’s take a look at everything that everybody else wrote. And I was trying to figure out how we would do that. Like, who knows how many internal blog posts there are and how many things that have been written about customer visits and things that different people know about users that they report out internally. And then I realized, I have some former academics on my team, academics who are now working as researchers in industry. They have skills in literature review. So, brilliant. I took them and had them put together like how are we going to review this and code it – special sauce, like they absolutely know how to do that. It was really easy for them to put it together. And then took other people who had more industry experience, or application focused experience, and said, “okay, now you can do what was set up. You can do the review of following the system.” And I have people who know how to set up systems set up the system.

Steve: So, that’s an example of the special sauce. They know how to deal with this.

Leanne: Yeah. And if you have a team who has mixed experiences like that, then they just sort of like, they lift each other up and they teach each other things. And that’s something that you can’t learn if you’re the only researcher, or you’re just one of a few researchers who all have exactly the same background. So, you all come out of the – whatever the programs are now, at Carnegie Mellon, Stanford and Berkeley and University of Washington and on and on. And you go and get a job and you do remote interviewing and some focus groups and some other things But everybody around you is doing exactly the same thing. Then you don’t get a chance to sort of like up level yourself. And you also, like if you’re the person managing that team, you don’t get a chance to learn new things from people on your team that you hired, that like you don’t have the same background as them. So, it just improves humanity basically.

Steve: So how do you think about like what the – if the team is kind of eclectic now and you have all these different mixes of skills and backgrounds and aptitudes, how do you think about what the collective should be? How do you think about the mix?

Leanne: Of a whole team?

Steve: Yeah

Leanne: So, I do miss the team I used to have which had both research and analytics. I think being able to have people on your team who do data science, or who are more like business analysts who know how to count things up and make charts, or someone who knows how to mix data together and come up with reports out of it, you get a much faster feedback between the analytics people and the research people. Analytics is like oh I did this, I want to know why? Research is like I can go out and find out why. Or research says, “I need some access to people who have certain characteristics, can you find me a list of them through analytics? Thank you so much, I’ll go out and talk to them and then I’ll come back to you.” In most companies those roles are separate. So, it is here. Analytics is a separate team. We work super well together, but sometimes when you have them both on the team it’s really nice because you’re both at the same team meeting and you’re both seeing what they’re doing. From a research perspective, having people who are all quant or all qual, or halfway in between or know both, is useful because anybody who’s in the job market right now, when they look for their next job the requirements are going up and up and up. We don’t want you to just be somebody who does interviewing. We need you to know how to design a survey. We need you do know how to do SQL queries. We need you to know how to do – like there’s more and more requirements showing up in job descriptions now. And so, if you can be a part of a team where somebody else knows that thing you don’t know, it just – we’re all going to like leave and get another job someday. Nobody stays somewhere forever. So, it will just help you when you’re going out to look for your next job.

Steve: So, skill development, kind of through proximity to your colleagues.

Leanne: Yeah. And it’s also diversity. Like any time, you have a diverse team, whatever you’re defining diversity as, just makes for a better team environment because all these different perspectives around the table from all the different places they came from, just makes for like different things to feed into feedback about a project you’re working on, or something that’s going on in the company. You know, we didn’t all come from the same school or the same program, so we don’t have a difference of thought in the way we approach a problem.

Steve: So, for you, thinking about the teams that you’ve put together – because you’re talking about different kinds of – I don’t know if diversity is the right word, the way you’re using it, but qual and quant are different orientations, or…

Leanne: And also, academic and industry. Or in tech, because I’ve worked in tech for 20+ years, I look for people to come work at the tech company I’m working at who haven’t necessarily worked in tech before. So, look for adjacencies. Like you did research, but you did it for a car company. Or, you did research, but you did it for an agency. So, that they can bring different things into the team. That’s what I mean by diversity. You know research, but you used to be a professor. You know research, but you were working on an astrophysics PhD and decided that’s not what you wanted to do anymore, and you realize things that you learned as a part of your graduate program – so you totally know how to do surveys and how to talk to people and how to do research.

Steve: I had this Twitter conversation with someone the other day and they asked me where is a writing course that I can learn to be better at writing the kinds of things I have to create as a researcher. They can’t find things like that. They can find persuasive writing which is for sales. I kind of scratched my head and said, “I don’t know that that exists.” I went into this little sort of pontification, as Twitter invites you to do, about learning from adjacencies like take a creative writing class. Take a journalism writing class. And in doing that, you would start to see, oh here’s how the lessons of doing this adjacent thing well could apply. That was the best advice that I had. I feel sort of affirmed, from what you were saying, that adjacencies for certain kinds of things bring a certain kind of value.

Leanne: So, the Luma Institute, who put this framework together around design thinking methods so that it’s easy to teach and easy to learn, calls that alternative worlds. And I think that’s a useful way to think about it, that for your person who is trying to figure out how to write a research report for whatever audience they’re trying to serve, well that’s storytelling. It’s sales, it’s marketing. It’s slide development. It’s all these other things. And when you start thinking of it as how would I think of this from the point of view of a salesperson? Or how would I approach this from the point of view of a marketing person who has to put customer stories on the web and sell that to somebody? Or how would I do this as the point of view of the person I am presenting this research to? Like what do I want to hear? It gets you out of the sort of like oh I have to have 10 slides and one has to have my methodology and the end one has to have like recommendations. You can start thinking about it more as like I want to tell this story so that they land on that naturally. Or, they ask me questions about it when I’m done that will answer all those things, so I don’t have to have it all in my report.

Steve: I feel like this is a transgressive notion of expertise though, especially in tech.

Leanne: Yeah. It would be nice for all researchers if they could just come in and do their research and research was valued and anything that researchers said they were learning from customers/users was taken and put into the roadmap. I don’t know anywhere that happens. And because there isn’t anywhere that happens and the way the culture in these teams work, you have to have this sort of like marketing and collaboration and partnership mentality around your work. If you’re a consultant, you get a project, you do it, you collaborate, you sell it a little bit – you’re done. You don’t have to still hang out and make sure it’s being used and follow-up.

Steve: It’s just all sunshine and roses being a consultant, right!

Leanne: Exactly. That’s why I don’t do it anymore.

Steve: I had a coach advising me, in my consulting business, and what they said was – they said many things, but one of them was anytime you start talking with a prospective client, show them – show me how you’ve helped someone like me. That adjacency kind of framing – the advice was that wasn’t as persuasive. So, that could be what have you done in my industry, in my vertical?

Leanne: Yeah.

Steve: You know, and I think researchers, or anyone that just likes to sort of – that gets deep into different kinds of problems, you know we see connections between things that are – you know we know why this problem is like this other problem, even though it’s different.

Leanne: And you want the other people to see that and sometimes it’s hard to show them a lens that will help them see that the way you saw it. Yeah.

Steve: I think the advice was that’s not where you start the relationship. You start the relationship a little more close in. “Oh, this thing that you’re doing. Yes, I have done that.” Even though every problem is new in ways that – it’s very hard to do that.

Leanne: Yes. Yes.

Steve: Sort of starting the – showing relevancy right at the outset of the conversation.

Leanne: When I ran a consulting agency, that’s how I started it. Somebody asked me if I knew how to do something and I’d never done it before and I said I knew how to do it. And all of a sudden, I had a consulting.

Steve: So, your career is just built on lies.

Leanne: Yes.

Steve: Is what we’re saying here, okay.

Leanne: Yes.

Steve: And we’re leaving this in. Yeah, well I think that’s true too. There’s the confidence to explore. I mean can you do this? Yes. Have you done it? Well, no, but I can do it.

Leanne: Yeah. And sometimes the question never comes up, have you done this? It’s just the can you?

Steve: So, that’s – so, we’re talking sort of requests in the consultant side of things. Inside organizations, in any of the ones you’ve worked in, like how are projects formed? Who decides?

Leanne: So many different ways. So, I would say that if you’re beginning research in a company, you’re taking requests and you’re probably using your boss or some other key stakeholders you recognize to decide which to work on. And you’re probably prioritizing those in some sort of backlog, if that’s in a spreadsheet or it’s a Jira or whatever. So, you’re showing people sort of like here’s what’s in my backlog. Here’s what’s coming up next. If you’re a team, the way I prefer to run a research team, which is the way I’m gradually getting it set up here – how it was set up at my last couple of jobs – is to have researchers who are paired or embedded, or whatever you want to call them, but they’re primarily working with a product area. A product team, a feature team, a product area, however your organization is set up. And they are servicing that team and partnering and collaborating with that team. And they own the research for that. But because they’re a part of the research team they also have a holistic view across the organization, because they’re also meeting with other researchers who are doing the same thing and seeing where their work, or where their product area or features sort of overlap, or interlock with each other. And then having some people who are more senior, principal researchers, just working on special strategic projects. So, that could be, depending on a company, that you have someone who is more focused on market research or competitive research. It might be someone who is focused more on research to understand where we should go next? So, sort of like future casting and sort of like looking at what’s going on, finding edge cases, seeing spots for innovations and other sorts of insights. And then also the operations side. People to manage panels of users and recruit people. And then there’s always, depending on how many people there are, some sort of management tier for people management and coaching and that sort of thing.

Steve: And when you talk about embedded you – it’s certainly a frame that you hear described with some regularity, but your words are a little different. Some people talk about embedded, that that researcher is on that team, sort of fully on that team. You’re almost describing like a workstream or a customer or something.

Leanne: Yeah, because the way people work on teams now is not the way that HR software recognizes. So, HR software, for example, recognizes I have a boss. He’s a VP of design. Recognizes I have people who report to me, all the people on my team. Um, maybe recognizes that I have people that might be good for 360 reviews. So, my direct colleagues who also report to my boss. But, that’s such a limited view of what I call my team because – I’ll just talk for me. Like I work with lots of different people around the company who, if I was going to be reviewed, I would want them to weigh in. People on my team – actually, we had a research team meeting this morning. I had them all bring a stakeholder map because I wanted to see – now that I have people that are sort of like embedded or paired within teams, like who are they working with and what does it look like, compared with people on my team whose work is sort of nebulous around all the different people that they’re managing and all the different work that they’re doing. And it turned out that way. So, if you’re a researcher on a team – a lot of people refer to the three-legged stool, but if you have a researcher it’s a four-legged stool. So, you have someone from research, from engineering, from product and from design, all working together with like a common focus on how are we going to get this out the door and make sure it meets customer needs, solving a problem, all that sort of stuff. If you are somebody in operations or a principal researcher working on a strategic project, you just have a mess of people around you. And so, all these people sort of like vacillate in between teams. Designers do the same thing. Designers are a part of the scrum team and they’re a part of the design team. Engineers are a part of a team for some sort of product or feature. They’re also a part of an engineering team. And if gets even more complicated when your manager is in one city and you’re in another city. Because if you’re enough time zones away you probably have somebody who is sort of like your dotted line manager in that city to help you with like all the things around your job. For example, I have a researcher in Barcelona. We can’t look over each other’s shoulders. There’s only two hours a day when we’re both at work. So, there’s a director of design in that office who is her sort of – who is her manager there. So, if she has questions about how does this work? Or how do I do this? Or who is in charge of this? Or help me review this? He can do that. But no HR software recognizes that. It doesn’t recognize she basically has two bosses. It doesn’t recognize that I have one boss, but actually these three other people are kind of my bosses too. It doesn’t recognize that that designer has a boss for whoever is running that scrum team, but that designer also reports to a design manager or director. So, most people work that way now. There’s very few people inside companies anymore who will say they belong to one team and that’s all they belong to. Everybody belongs to like multiple teams and we have this archaic structure around this person decides like your performance review and hopefully gets input from all these people. But, just lines people up in like lines and doesn’t realize that it’s a little more messy than that.

Steve: So, does that gap between the reality and the structure of the software, what’s – are there consequences to that for how we work?

Leanne: I don’t think there’s consequences. I think it’s just the way it is. I think there’s more 360 reviews now, just in my experience, then there were 20 years ago, because you work with so many other people now. I can remember jobs from early to mid-90’s. Nobody was doing 360 reviews of any of the jobs I had then. Like I worked with a set of people in a geographical space. I didn’t work with anybody else. I think it’s just more like it is what it is and things have come in to help manage and understand that, but we, the people who sort of like do to the sort of organizational management, haven’t figured out how to accurately represent that in a structure of the software that manages the company. Some companies try to go – what was it, like Zappos, that decided nobody had a manager. That seems like part of a reaction to that – Holacracy, that’s what it was called.

Steve: So, we got into this because we were talking about what does it mean to be – there sometimes seems a tension between researchers are embedded, or researchers are centralized. And I think what you’re saying is they are neither, or both. And that that’s how work happens.

Leanne: Yeah, and you have to look at the culture of your org. How does your org work? Who wants to participate and who wants – it’s that old RACI model. Like who wants to be involved? Who wants to be informed? Who wants to partner with you?

Steve: So, wait.

Leanne: Do you know the RACI model?

Steve: I know it, but I need to clarify it.

Leanne: It’s where you define – I might not get this right. It’s like where you define who is responsible/accountable, who is interested in consulting on it and who just wants to be informed. So, if you’re thinking about a project, you always have people who sort of fall into those categories. So, I just think it depends – some cultures are very top down and as you go up the ladder they only want to be informed and not as involved. Some cultures are more participatory and as you go up the career ladder, everybody wants to participate and know and be involved. So, it just depends where you land and get a sense of who just needs an inform, who actually wants to come along and sit with you while you do a remote interview, and travel 3 blocks away to visit the customer. You know how far up the ladder does that go?

Steve: So, coming into these organizations that have done very little with research to date – like understanding the culture, it seems like that’s got to be an initial step to get the lay of the land?

Leanne: Yeah. Who’s making decisions and what data are they using to make those decisions? Do they want any more data? Usually the answer is yes. Anybody wants to know more to make better decisions and, in a corporation, make more money. So, figuring out who those people are, where the gaps are, what they need to know. And then another thing you can do when you start at a company is be the researcher yourself of that org because what you’ll start to notice, just like any research project, is that you’ll start hearing the same thing from multiple people. So, you’ll start hearing like, “the one thing we’ve never figured out is whether people really like blue or red better.” You know there’s always something that’s like the one thing where people aren’t aware that everybody has the same question, but because you got them to open up and talk and had casual conversation and used all your interview tips and tricks to interview multiple, different people who don’t work with each other, you’ll start to see a theme around something that everybody wants to know, but nobody knows, but nobody is like trying to figure it out because they don’t know everybody else wants to know. I’ve seen that at every company. We’re working on a project like that here right now where there’s basically two things that I heard across the entire company when I first came here. So, I was like okay, that’s where I’m going to start some key research projects and then I’m going to bring in some people who are interested in it. And then as I hire people in, I’m just going to start attaching researchers to different product areas and work with the people who run design to sort of think about which designers need the most help right now. Or, what meets the business directives, or the key priorities, or the goals, or whatever – the product org of the company.

Steve: Are there stages of evolution? I don’t know if it’s milestones or some framework that at these points you’re kind of coming in where there’s very little and you start building in a way that is specific to what you’re learning about the culture, are there stages that you could identify that you’re passing through or moving towards?

Leanne: Oh, yes. So, once you get sort of like programs and tools set up you don’t really have to do as much anymore. So, programs and tools like is there decent survey software? Do we have access to users? How do we access them? Is there a panel? When we do research, do we have standard agreements that we sign with the humans that we’re doing research with? Repositories – where are we storing everything? How are we storing it? How are we accessing it? Do we need transcripts? Do we have a tool for that? Do we need to do card sorting? Do we have a tool for that? Just sort of like sorting out all the tools and processes for the practice. What was the question? I got lost on a rabbit hole of process and tools.

Steve: You were answering the question. So, what are the stages or milestones?

Leanne: Oh yeah, okay. So, looking at what’s needed that is more like capital improvements. So, like we need a new roof. Okay, we’ll get a roof, it will last for 20 years. Done. So, things like how you pay incentives and research agreements, that’s sort of like the roof. Then you need like everything that pays the electric bill and the gas bill all the time. So, those are projects. So, what projects need attention right now. And you learn that from interviewing sort of like all the key stakeholders you’re going to work with, which in my example earlier was once you interview a whole bunch of them you start to find out that there are some key things that everybody wants to know. And then figuring out, either with your boss, if she or he knows, or with key stakeholders, like if I was to start bringing in researchers one at a time, where would we start? Who would benefit the most? What would be most impactful? Or said another way, sort of like what are the things launching soon, or later? What are the business goals, or what’s in the strategy? Or what are the business directives? Or all those sort of things that make a company run. And that helps you because then you’re making sure you’re focusing on something that is actually going to eventually affect the company’s bottom line. So, when the research comes out and it helps that, then you’ve been like okay, this is how we do it at this company. We’re not academic. We don’t go sit back and sort of think about things and just, you know, go research things out of curiosity. We look at where the company is going to make sure we’re helping it and helping it do better with the research that we’re doing.

Steve: And then what’s a five year – that’s a presumptive question. They’re all presumptive questions. What’s sort of the horizon for which you would have vision? The time horizon?

Leanne: For the times that I’ve come into a company and there’s been no research, or one or two, like here, individual researchers, what I’ve told people is it’s going to take 12-18 months, with budget and with head count, to get this into a modern research practice. And here’s how we’ll do it incrementally and here’s how we’ll check in and benchmark and measure it along the way. When I was at Autodesk and I inherited a team and then I sort of hired in and added on to it, it was much different. These were really old products that had a really amazing install base. And some of them were market leaders. It wasn’t like companies that were two years old, five years old or ten years old, or fifteen years old. Like, once you’re at 30 years you’re settled in a certain way. So, I did different things. I tried to better understand the products and better understand the company to see where we would could add value. And then also, because that was a much larger company, see where research was happening across the company to see if we could sort of like combine efforts.

For example, I went to some company that’s like 25 years old, that has say 1,000 researchers. Because there are some companies out there now that have that many. And I was plopped in as a director of research with 20 to 50 direct reports. It’s a much, much different role than coming into the company and setting something up. They’re you’re coming in and you’re saying how is this done? Is there any way it could be improved? Is there any way I could sort of like do – no, there isn’t? Okay. So, how do we prioritize things? How do we have impact? How do we collaborate and share out and be good corporate citizens and be a part of the team?

I’ve realized, with this third job that I’m on now, after doing consulting for many years, that I do really like growing things. There’s a challenge to it that like gets me into work every day. Like, oh, we have to figure this out. Oh, there’s lots of problems to solve. Oh, how are we going to prioritize this? Or how are we going to solve for this? You know, how am I going to find someone to fill this role?

Steve: You just listed a bunch of things as positives that someone – if you’re in a different emotional state or a different sort of mindset – could frame those all as negatives.

Leanne: Yeah. Yeah. You have to understand the challenge you like to work on. Is the challenge I want to be the person who works with the team to help them figure out what they need to solve? Or do I want to solve an organizational challenge while also working on business priorities and solving problems?

Steve: So, this is something that kind of sparks you?

Leanne: Yeah. And I didn’t realize that until – it was like 3 months ago or so. We hosted a breakfast here and somebody was asking me what I was doing, and they were like, “Well it sounds like you’re really good at that because you’ve done that three times now, and you started your own company.” And I was like, oh, that’s the first time I realized like this is my thing that I like to do.

Steve: Yeah. That’s great.

Leanne: Sometimes you have to be almost 50 to have finally found yourself.

Steve: Right. It’s the journey, not a destination, right.

Leanne: Yes.

Steve: I’m sure you’re going to keep finding yourself.

Leanne: Yes.

Steve: Anything else that I should have asked you about that you want to share with us? The collective us. I know it’s just you and I in a room.

Leanne: Well, there is one other thing I’ve noticed in the past few years. I have used a community platform, that’s sold by a company and meant for marketing and brand teams, and have used it for product and design research. There’s something about – so, support always has its help forums. Marketing and brand usually do focus groups, but now sometimes they run online communities where you can get badges and win things for participating in their research, or telling stories, or whatever. There’s a space in between that people who are on research teams at companies sometimes use and sometimes don’t. So, a typical team might have a panel, particularly if you’re B2B and it’s hard to get access to people. Or even if you’re a consumer-focused company and you just want to make sure you have like your 10,000 people who you can access at any time to easily invite to research. But I think there’s something about the way that people on social media interact with each other in a group in one-to-many fashion and many-to-many fashion that is happening in some ways in research teams, and sometimes not. So, if you’re a researcher and you can, at any time, poke into a community that belongs to you – this is my panel, but your panel, they can’t all see each other. In a community they can see each other. And so, it acts a bit like a support forum, a bit like this marketing and branding, sort of like what you think of our new colors sort of thing. Or this is our new video for our marketing campaign. But instead it’s for product and design. I’ve noticed there’s a certain fear of it – oh my God, if we get our users and customers to talk to each other, what will they say about us? But I think if you get over that fear and you open it up and put together a private space where they’re all talking together and you and your product design, engineering, marketing sales, whatever partners are also in there, you create this sort of like brand loyalty and product loyalty that you don’t necessarily get just from marketing brand and sales and just from support. So, I’ve done that in my last two companies, created this sort of community where people could see each other, and we could interact with them in little like pop-up sort of projects. Haven’t done it here yet. But when I talk to the vendor that I’ve used for doing this, the last time they demoed it for me, I saw that the examples they were giving me were for a product and design team. So, I was like huh, you’re now selling to product and design teams. So, there’s something going on with people picking that up and doing that.

Steve: How do you set people’s expectations, the people that are participants in this community, for what – as you said, it has elements of other kinds of communities that are maybe more familiar…

Leanne: Well, you have to make sure, particularly in business settings, that – you know don’t share anything that you wouldn’t want your competitors to see, because you don’t know if your competitor is in this community, or not. We’re focusing on sort of like your work and your use cases of these sorts of technology. Don’t disclose anything that you shouldn’t be disclosing. But, if you’re in a space where people are using a technology where they get a lot out of finding out how other people are using it too, then you create relationships with each other and then you create your own relationship with them. Another downside is you can get bullies, so you need to have guidelines. We can kick you out if you behave in one of these ways. And with panels, because it’s the company to the user or customer, and that’s it, it’s harder to find sort of like an intrinsic incentive for someone to participate in research. In a community you usually don’t have to have any incentives that are hoodies or cash or gift cards or whatever because the intrinsic incentive is that they’re getting value from other people being in there. And then every once in a while being able to talk to a product manager or talk to a designer or talk to somebody else. So, I think that – I don’t know if that’s something that will keep going, but I see it as a way, particularly in B2B research, to provide value for your customers that actually helps out your sales people and gives your product people and design people a way to interact with users and then also give the user support that they’re probably also getting over in the support forum, but a different kind of support because you feel like you’re part of a group.

Steve: It’s compelling to me to think about just blurring the different kinds of interactions that companies have with people.

Leanne: Yeah

Steve: Like you said, there’s marketing, there’s research, there’s support and we structure the companies around those kinds of functions and the tools and so on.

Leanne: Yeah. And it can make it hard to see the end-to-end customer experience. Yeah.

Steve: What – maybe we can rewind. Like how did you end up in research as a profession?

Leanne: Back, when I graduated from college, in the early 90s, I went to work for a startup cellphone company as a statistician. My background is in statistical computing and economics from my university education. So, I was analyzing what were then considered very, very large sets of data – terabytes of data. I was running a neural net program on a Sun SPARCstation. I don’t even think it had a gigabyte of RAM. Yeah. It had two processors. It was amazing. It would take like 4 to 6 days to run something, to spit out a model. And if I got it wrong I had to do it all over again. What I was doing was I was working in a marketing organization and I was there to help them like figure out things like who should we sell call waiting to? Like should we just sort of try to sell it to everyone? Or should we target people? So, I was figuring out targeted lists based on behavior that we saw in cellphone use data, to help the marketing team understand who to sell things to. When they did that, what I realized was I didn’t know why the people were buying it? Like, just because I could predict like because you have these like usage analytics, it makes you more likely to buy, but why is that? And a couple of things happened then. The company I was working for got bought by AT&T, became what is now AT&T Wireless Services. When you get bought by a much larger company, a bunch of chaos ensues. Your job often changes. Your boss often changes. I was very young. I was in my early 20s. And at that exact time Mosaic came out and the web became graphical.

So, there were all these things happening that occurred at this point in my life where I was like, huh, this is my first job out of college, but now I’m interested in knowing why these things happen? I just taught myself HTML. I figured out how to put together a webpage. Oh my goodness, there’s this thing called Match.com and I met this girl and she lives in San Francisco and I want to date her. Oh, I’ll just go take this job at this company in San Francisco, this web company that has like 20 people and they’re making websites and they’re from all different sorts of places. And I became a QA manager there. Once again, sort of being asked, “can you do this?” Not, have you done this. And I said, “yes.” And then from QA realizing that we didn’t really know what the users were doing. And I, at this point, had never heard of user research. But I was friends with people who were doing that at different companies. So, I asked them about it. And then I left that company and went to a startup. Was also a QA manager. Got laid off when the company didn’t get its next round of funding and went and did some consulting. Doing like web development for the summer. Set up a company that I had intended to be a company that did QA for web apps and somebody came to me and said, “can you do some researcher interviews with users while you’re doing browser testing?” And I said, “yes, we can.” And I was like here’s my chance. I’ve always wanted to know why people are doing things. And so, I happened into it that way. And in the mid to late 90s anybody could do that. You could teach yourself PHP. You could declare yourself to be a web marketer. You could say I’m an information architect. Like you just said what you were, and you could just do it. So, I quickly went and found some friends and said, “I have this consulting project. I don’t know what to do, help me out.” And they were like well you do AB…like the things I just mentioned. You have to do A and B and C and D. I was like okay. So, I did A, B, C and D. I did the project. I got paid. I thought it was amazing. I was like I want to do this again. I was like a kid who gets ice cream for the first time. Like, I will have more of that. And that’s how it got started. In consulting, what you need, from my experience, is you need one or two projects as reference projects and then that helps you get the next project and the next project and the next project. And then it was 17 years later.

Steve: Wow. You’re saying that there was a time when you could sort of declare yourself to be this thing.

Leanne: Yeah. When the web was new. When this new technology was coming out and consumers were grabbing onto it and companies were seeing that money could be made there and – people think now that there are more jobs than people, but then there really was more jobs than the labor market could supply because the labor market didn’t have the skills and the education system wasn’t educating people with those skills. For example, in looking for research interns for this coming summer, I interviewed people who were getting degrees in user research. Like a Bachelors of Science in user research. And I was like wow, like companies are now preparing people and there’s a whole education system out there in these D Schools and I Schools and everything now that didn’t exist in the 90s.

Steve: Right. I sometimes feel like oh I’m – ‘cuz I came up through the same system, or lack of systems, that you did.

Leanne: Yeah. If I wanted to do this job now – say, if I had gotten my Masters at the Berkeley I School, or whatever, and I went and got a job as like a senior researcher at a tech company, and I worked myself up, way up, it would be much, much different than my experience of teaching myself and figuring it out.

Steve: What’s lost or gained in that evolution, if anything?

Leanne: Well, I can clearly see that my sort of like – YouTube didn’t exist then, but it would have been like the university of YouTube, the way that I went about teaching myself how to do research and starting up a whole consulting agency around it – it would have been much different if I had been schooled in a particular way in how to do research. I think some things would have been lost because it wouldn’t have been sort of like me and a couple of other people just sort of figuring out how to do something. You’d come up with new stuff. But I think some things might have been gained. I might have more confidence – like I have a Masters in this, so I know how to do it. Or, I did my thesis on this and I am – yeah. There’s a certain amount of imposter syndrome that goes around, even if you have that education. But sometimes when you don’t, and you make it up yourself, you can doubt whether or not you really know what you’re doing. Fake it ‘til you make it.

Steve: I remember just being in a consultancy kind of around that period of time and that every prospective client that came in we would have to write a proposal for what we were going to do. And it just felt like writing a Master’s thesis every time. Or some just impossible thing to give birth to. Like explaining what it was, deciding the steps, trying to articulate those steps. We just didn’t have any point of reference and so we were sort of figuring it out every time until I think at some point we started to settle in and like oh, here’s kind of what – what it looks like. And now like writing a proposal is not hard. There’s many other hard things.

Leanne: But now you have templates and you can copy it.

Steve: And you know – I mean I have a narrative that runs through my head that oh yeah it kind of goes like this. We have more tribal knowledge as well as…

Leanne: Yes. More people to reach out to, to get examples from too, than used to exist.

Steve: Right. In this era, this consultancy, not only were we figuring it out, we were doing so in pretty much isolation. Because you knew a few people, but most of us at that era – there wasn’t a community of researchers to kind of connect to.

Leanne: Yahoo Groups did not exist yet. Google Groups was not there. Google wasn’t even here.

Steve: Yeah. This is great. It’s a very interesting conversation. I learned a lot.

Leanne: It was great chatting with you Steve.

Steve: I appreciate you taking the time.

Leanne: Absolutely.

Steve: Anything else? Are we done? Any last thing?

Leanne: It’s fantastic. Yes. Perfect ending.

Steve: Mic drop. Alright, thank you.

That’s it for this episode of Dollars to Donuts Go to portigal.com/podcast for the transcript as well as links for this episode. At the website you can also check out all the other episodes, and subscribe to the podcast on iTunes, Spotify, and all the usual places. My books are available at Amazon and rosenfeldmedia.com. Our theme music was written and performed by Bruce Todd.