I recently did a small Ask Me Anything session organized by Mind the Product. The video is now online.

It runs about an hour and includes discussion about balancing quantitative and qualitative research in an organization that leans heavily towards one or the other, doing research when you have an exceptionally small user population, what happens when the work you are performing is well outside your job description, and more!

I recently did an “Ask Me Anything” session on the Rosenfeld Media Slack. We touched on subjects ranging from how to handle difficult clients, my favorite band, and recommended reading, to dealing with “heavy topics” in interviews and how to improve your skills. Here’s a recap of the session…

Q: What book except your own would you recommend to read for UX designer? -Natalia H.

A: There are so many things to learn about. For design, I think this recent book by Scott Berkun is a great examination of how design is everywhere and in everything. It’s a quick read, it’s fun, and it’s empowering for designers I think, telling us again about all that we do, we have done, we can do.

Q: I often deal with “heavy topics” such as life, disability, cancer etc in my line of work, so it is not unheard of that we have someone break down and cry in interviews. We try our best to mitigate and avoid unnecessary stress (participants health and safety is our primary concern and we tell them in advance about the topics that we will be discussing), but can you talk about some advice how to mitigate these situations or what to do when this happens? -Fabian B.

A: Of course the stress and the emotion is going to impact all the people involved, say the participant and the researcher. I think researchers need to keep in mind that they aren’t (in most cases) trained for this, and that they need to find ways to take care of themselves. Often we will emote less in the interview than we might want to, so we want to leave space for ourselves to have feelings, to have reactions, to have someone to talk to. Work out what that is going to be. Who are you going to be able to speak with? When will you be able to speak with them?

For the participant, I learned something new to me from Sarah Fathallah at the Advancing Research conference when she talked about referral paths, something that if you are doing academic research that has an IRB ethical review needs to be put in place (someone correct me if I have this wrong) – where you identify things that might come up, like if someone reveals they are being abused, or is going through addiction, or having suicidal thoughts, you already know what your action is going to be.

I don’t know that we need that level if we aren’t doing say specifically traumatic research, but it’s the idea of planning for that. I’m also not saying that we need to DO anything; I think well-intentioned but under-informed do-gooderism is potentially worse than doing nothing in certain situations.

The complexity here is you can’t come up with every possible thing that might come up. But you can come up with SOME. I am also intrigued by sort of a generational shift in how we see our role; from “we need to observe, listen, empathize, not judge” to “all those, but we must also help.”

Q: I suspect a lot of us always did at least some of our research remotely, but now with covid, we’re seeing way more remote/virtual research. What are a couple of your top best practices for doing remote research well and getting high-quality information from respondents? -Amy B.

A: I think it’s worth co-opting that thing I see tweeted all the time “you aren’t working from home, you are working from home amidst a global pandemic.” Same for research, right? All the parties are living through an emergency. So yes, there may be dogs and kids, and construction noise, and someone may be in their garage so they can get some privacy. And as we’ve adapted our tolerance for informality in terms of focus, energy, duration, attention in work, we can apply those to these interactions with strangers. A guy I spoke with yesterday put us on pause as his daughter asked for the car keys, she just got her license the day before. It was fine. I didn’t get ruffled like I might have in another time (no that’s crazy I’m nothing if not entirely cool and rolling with the changes). I don’t have a good answer for how fully remote research locks us out of getting to certain people. I just had a brainstorm with a colleague who was thinking about how to understand how people were or will be using transit, given that we can’t go on transit right now, and that the people who he’d want to learn about are possibly not sitting at home with a room with a laptop and a webcam and a good internet connection that is available all the time for them to just get on a call with us over Zoom. So who are we excluding even more so now (as society is excluding people in those circumstances even now) ? I know people are trying hard to deal with this, but I don’t know?

Q: Which musical group has had an interesting impact on your life and how? -Corey B.

A: I’m going to say The Tragically Hip. Being an expat Canadian, when the Hip would come to town it was always an interesting gathering of the community, at least a bunch of fellow Canucks. You’d go to a show and see hockey jerseys and University leather jackets, and just stuff you don’t EVER see in the US. And as time passed, we all got older, the fans got older, the band got older, and how we expressed ourselves and our identity shifted, gradually. I haven’t really moved through an era of my life as clearly in common with other people, not close friends, just strangers, but to feel it so tangibly through ANOTHER experience has impacted me. And of course the ultimate end of aging is death and the loss of Gord Downie still impacts me every day pretty much. I listen to the band and my many bootlegs and think about them and him, constantly. Perpetually.

Q: Do you have any tested tips/tricks on working with difficult clients? I recently had a challenge of convincing the client to use our approach when they wanted to do something else. Curious to hear what you do in such situations. -Kama K.

A: I don’t know enough about the context, and the power dynamic, as I think it often comes down to that. Two big directions, though: Empathy/Walk away. For the first, and I don’t mean to tell you stuff you already probably know and do in all your relationships, but there is that weird power to detoxify some broken interactions when you just listen, when you ask more, when you acknowledge, etc. Probably this has some buzzword for leadership people, I don’t know what that is. It sets you up to say authentically “Yes, And, and not No, But.”

For the second, if someone is a client, then you aren’t in a full-time job with them. Of course as someone who also provides services to clients, I want to do a good job, I want to be appreciated, I want to bring value, I want to be employed, and brought back in, etc. So I don’t like that framing that design people in agencies sometimes like to espouse which is making the client the enemy. But we can walk away. And even if we just hold in our head that we have the OPTION of walking away can making a less desired choice perhaps more tolerable. I’m CHOOSING to do this which I don’t agree with and which distresses me but I could and may at some point choose NOT to do it.

Q: What are your most counter-intuitive insights about research as it propagates through the product development process? -Scott W.

A: I don’t know that I can claim counter intuitive as a goal, if you agree with me then it’s probably not THAT counter intuitive? I think there’s this desire to create models and visuals that say if you are here do this, if you are here do that. Whether that’s the double diamond model or (VERY HELPFUL) recommendations for methods based on a stage in a process like Christian Rohrer’s.

I just am not that organized, I am just not able to constrain myself to a discrete stage, maybe I’m trees and that’s the forest and I just can’t do the forest well enough. That all being said, I think there’s something about how these processes are a continuum, a gradient, and not stages. I know Software Development Lifecycle Methodologies are all about gates, and stages, and review cycles, and (ugh) sprints, but I think in terms of where we are in terms of certainty, of belief, of ideas, it’s a much more creative process, I can’t control what thoughts I have in the shower in that way we have of considering and being inspired about what it means, what to do, what the opportunity is. Some of that is always happening for creative people – and research is ABSOLUTELY a creative process of sense making and understanding.

Q: Are there any topic(s) or technique(s) you hope more places of instruction cover for individuals entering into research? -Randolph D.

A: As you know, I love storytelling, and I think it’s such a powerful tool, but it’s just kind of considered to be something that is maybe part of your personality, your own personal toolbox. But obviously it can be taught, practiced, developed. Research is about gathering stories and creating new stories, and “story” is of course an ENORMOUSLY broad construct so one can take it however one likes.

I mentioned ethics in another question, I think as a field we haven’t reckoned with it sufficiently. I’d like people coming into the field to have a perspective on it. I know a researcher who wrote up their own research philosophy, not even as a document to publish but as a way of working out what they were trying to do. I was extremely impressed with that.

Research as a practice to me is a constant consideration of who we are in the world, as people, how we relate to other people, how we judge and don’t judge, and just how we move around and exist and perhaps make and help. So, being intentional about what you believe and what you want, damn. It’s a brilliant activity. I haven’t ever done that and hadn’t ever thought of doing it. So philosophy isn’t ethics literally but is adjacent?

I’d like something in training – and I don’t know that the places people are learning about research are the right environments to be considering this aspect – about what the researcher’s role is relative to the rest of the people they work with. I do not like the idea of research as “support” – you hear “oh I support three teams” – I know research is a helping profession like say librarians, and I don’t mean to squelch that strength, but I think we are partners and leaders, and if we don’t believe in the value we can bring in the role we can play, no one else will. I think it’s a hard field to break into and there are a lot of entry level people and so if they are told they have to be subservient, that can set a long running pattern for their career and for the practice.

Q: Someone interested in working with us recently asked how we recommend she improve her interview skills. I recommended your books, but are there any classes or seminars you might recommend as well? -Amy B.

A: I’m going to pitch my upcoming workshop. Also very very good is the cycle of “do, observe, reflect.” (which probably has a smarter name than I’m giving it). Listen to Terry Gross – listen to her technique. Reflect on her technique. Print out a transcript and mark it up. What choices did she make? What other choices could she make?

Listen to your own interviews. Print them out. Listen to a colleague’s interviews. Same same. Have someone ELSE listen to your interviews. Have someone else mark up Terry Gross (or anyone who does a lot of interviews).

I think training will get you further than you are, but practice, man practice is the way. Do a lot of interviews. A LOT. Reflect and analyze!

Q: What are some tips for building rapport in remote interviews? (with camera and without camera) -Erika

A: One avenue to explore is pointing to the medium, just acknowledging that you are doing what you are doing. And not pretending that you are as smooth as you are when you are in the same room. I did an interview yesterday where I had to share screen from Google Slides (not what I normally use for giving talks) and see speaker notes and it’s just a mess, and so I stopped and said what was going to happen, and then you heard the “unprofessional” sounds of me, saying, “Okay I’m going to hit share screen…yeah I think it’s shared now, okay, now I’m doing this, can you see this?” It just normalizes the interaction so you are both having a similar experience.

One thing I think needs to be explored is around shared sensory experiences. I saw Alice Waters talk recently and she described how she’d meet with people and she’d put a piece of fruit or something else down in between them and they’d just eat it beforehand and it created these interesting connections and well, rapport.

I don’t know how to operationalize this for remote research but I’m imagining having everyone pet their furry animal before starting and just sharing that moment that is about the senses, even though we are having our own experiences, we are having similar ones together. I think there’s probably some work to do to create that in a non-weird-sounding way.

I did an Ask Ask Me Anything with What Users Do recently. Here’s the recap they put together.

Any tips for interviewing users who work with highly confidential information? Any strategies for incentivising such an audience to be open about which problems they struggle with and how they use our software, when almost everything they do is supposed to be a secret? [Timi Olotu]

I think it’s always good to set expectations clearly when arranging for interviews; perhaps in that situation there might be concern – before even agreeing – about risk. I worked with a bank and they had a standard set of text they used in recruiting bank customers, along the lines of “We won’t ever ask for your bank balance or any information about accounts.” Perhaps that was required for regulatory compliance anyway but it served to be very clear about what we were NOT going to be asking about.

I think managing expectations is more important than incentive; but gaining access is also about understanding their dynamic or their relationship to you. Like, if they use your software, then they have a chance to give input and feedback.

I’d also add, I wonder what kind of anonymized experiences you can create, and I don’t mean that to be so fancy sounding, but how much of the interview could you do on paper, with wireframes.

I don’t mean give them a usability test, but can you give them a set of high-level scenarios and have them pick some and choose which to walk through using a sort of simulated or high level version of your experience.

You wouldn’t want to do that for the whole thing, but it could be a component of an interview.

When generating questions for interviews, how do you reduce bias from pre-existing hypotheses? [Amy]

I think bias is obviously a concern but it comes up so often as the main thing people are concerned about. This is totally biased work! We are humans who are the product of our experiences and meeting other people and exchanging the slipperiest of substances: words! How can we not have bias?

I think that having hypotheses is a great thing when going into research. I mean, you put it exactly right – hypothesis. That’s not a closely-held belief, or an aspiration, it’s an idea of what you think might be happening. That sounds like what we’re supposed to be doing.

If there is a belief or a hope or an expectation – be it implicit or explicit – that seems like something we want to get to in research.

Of course there’s a ludicrous way to do that. “Don’t you agree that it’s better now that we have this feature located in this part of the UI?”

That’s a biased form of inquiry, that’s almost abusive of your power. But trying to understand someone’s framework, expectations, preferences, experiences, mental model, etc. from an open and curious point of view, and having that curiosity informed by what you have been already considering about the problem space, sounds like good research to me.

You writings and talks have been a big influence on many folks working in the area of UX – but who are your influences and inspiration, and why? [Rob Whiting]

Our field is packed solid with great people. I keep thinking about some of the over achievers I get to meet and how accomplished they are.

I love Jess McMullin, he is one of the first people to start doing civic design, like YEARS ago, before it became such a big thing in so many parts of the world.

I am a big fan of Allan Chochinov (he wrote the foreword to Doorbells, Danger and Dead Batteries). He started a graduate program in design at SVA in NYC, called Products of Design that kind of takes a big picture look at what design can do. Mentor, friend, inspires me.

I think Kevin Hoffman is really inspiring. Funny, passionate about pop culture and knows everything, cares about people. His book about meetings is going to come out soon (I can say I saw the cover and it looks really cool).

It’s horrible to try and pick people as I’m using recency to come up with a list.

I’m trying to get user testing ingrained into my company’s process but it’s a struggle. It always seem to be the first element dropped when budgets are tight. Any advice on keeping it a part of the process? [Mike Mellor]

I think you’ve got the #1 FAQ about research… that it isn’t supported. I don’t think there’s a stock answer though. But you might wonder – or seek to understand – why is it being dropped? And why was it even being proposed or considered in the first place?

I mean, it’s one thing to say there’s no budget or time… but if someone chose to put it in to begin with, there was a narrative about its value.

If one could understand that better, one could propose an alternative or advocate, with a bit more information. I’m sure there’s some Rhetorical Studies model here I’m not expressing well, but understanding your adversary’s objections seems like one possible persuasive technique.

I’ve long said don’t advocate for the process “we have to do research” but for the outcomes “we have to make sure we understand this issue or this consequence will happen.”

If you want to get into a discussion of timing, you can use the above diagrams as inspiration – lets say the top one is sort of my gold standard, here’s what it takes to do it “all” – but if you want to do it more quickly, here’s how it’s going to look – it may be more or less appropriate but that way you can have a discussion about tradeoffs.

If we only have a day to find participants, for example, then we can’t be too picky, we can’t go beyond who we know right now at this moment. Maybe that’s sufficient.

So even though your question was about doing it – or not doing it – I think looking at ranges of commitments – where zero is in that range – and encouraging reflection on trade-offs – could be good. It’s not about what YOU need, it’s about what the work requires. So don’t take it on yourself. “I’m not allowed” “they won’t let me” – it’s about us, about our shared goal and your expert advice about how to reach that goal.

I’m wondering what strategies and methods you use to analyse data from your interviews. Would you recap after each interview and write down your observations when they are still fresh and then wait for all of them to be over to listen to transcripts? [Edyta Niemyjska]

You describe my preference pretty well. I separate the “processing the experience” and “processing the data.”

After an interview, I might do a debrief worksheet – but typically not. But at the end of each day, I write up a VERY quick paragraph or two about the interviews we’ve done.

It’s meant to be a storytelling exercise, it’s a forced analysis (take a large thing and pull out some smaller bits) – and it shares the fieldwork with the rest of the team. Here is a PERSON, they have a NAME, they own a THING, they told us an EXPERIENCE. So it helps me make a first pass at distilling and it gets people to think about these real actual people really quickly.

It’s not field notes.

I try to do it in just a few minutes, and do it stream of consciousness.

When we’re done with fieldwork, I like to sort of collate, very quickly pull together a topline – here’s what we think we’re hearing., what did you all think, what did you all hear. We started with questions and we have some thoughts, we have some weak signals, we have some things we’re excited about. Nothing about what to DO with this info, just where we’re at, at the moment.

And then, finally, let’s dig into the transcripts and see what actually happened.

What is the most effective way to ask simple questions to better understand where our users are coming from? Often, the users are ill at ease and want to “help” or are simply biased so they muddle the actual answer. [Karunakar Rayker]

Part of your question is about building rapport. People are often ill at ease at the beginning of a session. They want to do a good job and they don’t really understand what is going on, I mean not to be patronizing, they understand, but they don’t “get” what this exchange is meant to cover.

It’s one reason why super short interviews are challenging because it’s hard to get to a point in the relationship where you have established a smooth dialogue, where the person is not only comfortable but excited, reflective. That takes time, sometimes a huge amount of time – and people are unique, and the way we find a connection is unique to the combination of them and us.

The ways we have to contact with people ahead of time – before the interview – can be rapport builders. Maybe we have a quick phone call and let them ask any questions. Maybe we give them an exercise so we can see something about them. Exercises also prime people – it engages them in thinking about the topic so when we meet they aren’t coming up to it raw and fresh and new, it gets them involved.

Sometimes we make sure in our recruiting process that we screen out people who aren’t already meeting a certain comfort level – “the articulation screen” – if the person can’t answer a question from the recruiter (tell us about a recent experience you had going to the movies) for a few sentences, then they may not be the best participant for the study.

But assuming they are “articulate” – they may not be comfortable. So our job is to keep listening, to keep affirming. I do NOT mean “okay great! Cool! Wow!” etc. I mean listening, I mean, asking follow up questions, expressing interest, validating that their point of view is important because you give it time – that’s a harder way to validate the person because you want to do MORE, but when you do that enthusiastic thing you are actually pushing them to perform for you.

Finally, when you have an uncomfortable person YOU feel uncomfortable, you are sensitive to the cues that this person is feeling weird and I should probably do something different or leave. What if you could ignore those cues – which are about YOUR feelings? And just keep listening and focusing on them?

Can you share examples of what type of exercises you have the user complete when connecting with them prior to an interview? [Anne Jackson]

Come up with something that seems relevant, and so many different ways to go about it. But an approach can be ask them to take a picture of two different things, and send the pictures along with an explanation. Two, because it’s about examples of contrast.

Send us a picture of something in your neighbourhood that you think adds value to the experience people who live there have. And explain why. And send us a picture of something that detracts from the experience.

Send us something you’ve organized well. Send us something you wish was more organized.

These are kind of digging into the theme you suspect the session will get into.

The interview kicks off by getting them to tell you that story again!

It could be fill out a form and give a couple of examples, but the photo stuff can be fun. A screen shot, even.

I am currently in the process of introducing a lot of PMs and Engineers to customer interviews. We are also training a few younger designers to talk to their target users.

What are some basic strategies and tips to keep in mind when introducing non UX researchers to UX research? [Nachi Ramanujam]

I write – well, scribble – on the paper. I might draw a big circle around the quote that is interesting and then write my own thoughts, “Why does she do this” or “they don’t have alignment between their goals and their choices” etc.

This formative UX study is a bit more tactical but might be really helpful – it’s so well explained (not specifically transcripts but at least about analysis in general – I think less about synthesis – where we take small bits and put them into new ideas and frameworks – which is what I think we do with the big mess of annotations I’m producing).

My consulting guide to fieldwork is a one or two pager that is meant to help people do well when they are joining in (NOT LEADING) user research interviews. It is the most boiled down set of points I have. I think it’s like anything, the more you put in, the more you get out.

Here’s a 40 minute presentation that is about doing research. Do they have 40 minutes? https://portigal.com/speaking/ has a bunch of links to past talks so you can see where there are videos and slide decks.

I also do a workshop where I ask people to interview each other and then reflect on what worked and what did not work. Practice – in a safe place – not on a work problem but on a practice problem.