Thanks to Chad McAllister and Product Mastery Now for their interview with me, titled How product managers best interview users (bonus: it’s still good advice if you aren’t a product manager)

You can find our 35-minute conversation on the episode page (and on Google) and embedded below.

Updated: now on video

Summary

[2:52] Why did your book, Interviewing Users, need a second edition?
It’s been 10 years since the first edition was published. The fields that we all work in have changed. There was a little bit of discussion 10 years ago about remote user research, and now remote research is much more common. I wanted to talk in-depth about the best practices for remote research, even as they’re still emerging. Research operations, which is a field adjacent to user research, has emerged. The book also draws from 10 more years of me doing research and teaching research. I’m always learning. I updated the stories and included better examples.

[6:09] How do we ask customers the right questions?
First, don’t assume you know what people want. Second, recognize that just asking customers what they want is not effective. There are a few related questions that you should answer.

Business challenge: What do we want to do? What do we want to change? What’s coming up? Why are we doing this research?
Research question: What do we want to learn from people?
Interview questions: The questions you ask customers.

What you want to learn is not the same as what you should ask. For example, if you want to understand where people find the most value in their budgetary spending, don’t ask, “Where do you find the most value in your budgetary spending?” Instead, craft a set of questions and build a discussion guide that has a flow and sets context. Ask questions like:

• What do you do?
• How do you do it?
• How long have you been doing it?
• What are you big problems?
• Where does budgeting fit into those larger problems?

Use the interview to ask many questions to get a larger context so you can conclude what the answers to your research questions are.

[10:19] How should we prepare for a customer interview?
Once you understand your business question and research question, think about your sample. Who are you going to talk to? Be creative in your sample. Don’t talk to the same people over and over again. Be intentional about who is going to give you the most information. Talk to people who will give deeper insight about the situation so you can make decisions about the changes you want to make. Figure out who will give you answers to your research question.

Next, figure out how to get to those people.

Then figure out what you’re going to ask them. Write a discussion guide. No interview looks like the guide you write, but it is a great tool to share with stakeholders to respond to their questions. This is a case of “Chance favors the prepared mind.”

[12:42] How do you avoid interviewing the wrong sample?
It’s much better to have a stakeholder object to your sample at the beginning of the study than at the end. Have a rationalization for the people you are choosing to talk to. Make everyone aware of the tradeoffs and how far these interviews will take you in your understanding.

[15:13] What do you do during the interview?
Don’t go through the interview line by line. All the good stuff comes from follow-ups. In the best interview, you ask one question and everything else follows. Every question you ask comes from something they just said. You are connecting and engaging such that everything they say is important because you have another thing you want know that builds on it. Telling someone honestly that their information is important and valuable to you creates rapport that makes these interview successful. Asking follow-up questions is a great rapport builder. You’re trying to poke around in the dark with a flashlight to see what’s behind the corners. You’re looking for your own understanding, not just checking off questions. Follow-up questions come from your own curiosity.

You can’t really do an interview by starting with a single question. There are points where you have to switch gears. That technique is really important. I call it signaling your lane changes. Tell someone, “Well, this is really great. I want to switch topics a little bit here and now move on to your procurement process.” Again, this tells them that what they’re saying is important.

[5:35] How do we stay curious and avoid developing biases?
Cognitive bias is very natural, and I don’t want anyone to feel bad about it because I think feeling bad about it makes it harder to overcome. When I do an interview and uncover one of my biases, that’s the most fun feeling. You could easily feel stupid because you have biases, but I think we’re doing this research to learn things so it’s exciting when someone knocks our sandcastle down. Not every bias you overcome is an insight about your product, but it puts you in that mode where you realize you’ve been holding onto something. Be able to hear when you have biases and feel good about that.

The skill is not to not have biases. The skill is to be able to hold on to multiple truths at once. Set your worldview aside and use the interview to embrace somebody else’s worldview. Set the intention that makes you curious.

You want to hear how they approach their problem or your product. That doesn’t diminish what you know. Being curious and having a beginner’s mind does not negate your expert’s mind. You just compartmentalize them for a time during the interview.

[24:42] How do you ask questions?
I like to have a first interviewer and a second interviewer. The first interviewer controls the flow of the interview. The second interviewer records the interview, listens deeply, and identifies things to be curious about that have gotten past the first interviewer. Before transitioning to a new topic, the first interviewer asks the second interviewer if they have any questions.

[27:35] How do you analyze data from interviews?
In general, you might do two hours of analysis for every one hour of synthesis. Analysis is taking large things and breaking them down into smaller ones. You can pull out some things from an interview about the person you’re interviewing. These are not conclusions, just distillations.

Synthesis is taking small things and organizing them in a new way to make something larger. You reorganize the broken-down pieces from many interviews. This can be done with affinity maps. You start creating frameworks that create segmentation or a list of priorities.

How much analysis and synthesis you do depends on the size of your project. Don’t just tabulate what people said—you’ll miss all the nuance. You’re creating your narrative, a new story that’s put together from the data.