Joel Kashuba has practiced design for nearly two decades, with a career spanning the practices of architecture, industrial design, branding, UX, and innovation consulting. He currently leads the Innovation & Design functions for Fifth Third Bank located in Cincinnati, Ohio. He told this story on stage at Midwest UX 2017.

While working for a major CPG company I was placed with a cross-functional innovation team assigned to write and vet concepts that would take a well-known women’s shaving brand into several other personal care categories. The focus was on serving the needs of young women in several BRIC countries. The theme we had been asked to unlock was “A Day at the Spa” – a theme the company had uncovered in earlier research within the United States and projected as a fruitful area to mine for opportunities and frame our expansion.

Before going out into the field – specifically, to Moscow – the project team undertook countless hours of concept writing sessions, often with heavily resourced vendor partners. We created roughly 25 concepts, each taking unique inspiration from the theme “A Day at the Spa”. Armed with our concepts we set off to Russia and began collaborating with consumers in the field to vet each concept.

By the noon on the first day, none of our concepts were resonating and we recognized our first challenge. The translator we had been assigned by a local agency was an older Russian gentleman who sounded much like a James Bond villain. As he readied each of our painstakingly word-smithed concepts, they each ended up sounding like the dastardly ideas of a dour old man who may like to cross-dress. To fix this, we recruited a spritely young woman who worked as an assistant concierge at our hotel to read the concepts. She was great! Several of our consumers even mentioned that she had the perfect voice for commercials in this category.

Despite this change, our concepts still weren’t hitting the mark we were aiming for and we couldn’t figure out why. These concepts had been exceptionally well received in our early test back in the States – what was going on here in Moscow that made them such tankers?

Finally, near the morning of day three, one of our consumers asked us plainly, “Why are you trying to make me feel old?”

“Old?” we asked her with sincere confusion, “Can you say more? Is there something in the concepts that makes you feel old?”

“Yes,” she quickly retorted, “you keep talking to me about spending a day at the spa.”

“And what does that mean to you?” we had our translator ask her.

She looked surprised and a little pissed off. She explained, “It means the place we send our grandmothers when they are too old to take care of in their homes. It’s the place people go before they die.”

It hit us like a ton of bricks. In Russian culture, a “spa” is what we’d call a retirement home. As we had been pulling out concept after concept trying to get these young women to fall in love with our theme, all they saw was of tone-deaf Americans shoving the idea of products for a retirement home down their throats.

We were horrified. We called off the rest of the day’s consumers and stayed up all night re-writing the concepts. The young concierge we had hired to translate became an adjunct team member. Constrained by time, we changed our strategy and turned consumer research into consumer co-creation. We had consumers work in teams to read and re-write the concepts, which were passed along to other teams of consumers to be refined. By the time we finished we had three great concepts that all resonated well.

Coming back to our home base, we reflected on the experience as a team. What we had set out to do was valid, but how we remained nimble in the field is what made the clear difference in how we would found success.

Nadav Zohar is a UX researcher at AEP in Columbus, OH.

My first ever user research project was for a healthcare app. Our users were nurses who work with poor and high-risk patients, often called “the under-served.” My supervisor and I had a reserved conference room at the client’s site, and our pre-scheduled users rotated in about one per hour. It was a grueling two days of nonstop interviews. For the first day I took notes while my supervisor moderated.

On the second day, after he moderated the first couple of interviews, my supervisor turned to me and asked if I thought I was ready to take the lead on the next one. I said “Sure” so he handed me the discussion guide. In came our next user, a middle-aged nurse who was very sweet and eager to help us in any way she could. This was my very first user interview and I was ready for a clean, uneventful affair.

As the questions on my discussion guide turned to the technological hurdles she encounters when helping her patients, her frustration mounted. At one point, while discussing how her technology failed to help her manage the stress of the enormous workload placed on her and her colleagues, she mentioned having lost a patient. I watched her relive that pain – she broke down and started sobbing. None of the other users we’d talked to had even come close to that kind of emotional response, even though some of them had lost patients too.

Right then and there I learned there’s an awkward balance between not wanting to seem clinical and cold at that crucial moment, but still wanting to preserve an interviewee’s dignity: I figured weeping in front of strangers at work must be somewhat embarrassing. So I bowed my head and looked down at my notes, or my lap, or at nothing in particular, to give the crying nurse a bit of privacy. I waited a few sobs so it didn’t seem like I was trying to shut her up, and then I warmly and gracefully offered her a box of tissues. I let her know I empathized with her pain (although looking back on it I don’t see how I really could have…but my empathy felt genuine anyway) and she eventually calmed down and we finished out the interview. After that, back at the office I was jokingly known as the guy to call in to make people cry.

I think I deal fairly well with very emotional user research situations and over my career I’ve learned they are not uncommon, but it was interesting to have one right off the bat.

Emily Mayfield (Twitter, LinkedIn) is a User Experience Researcher at The Kroger Co. in Cincinnati, OH.

Before my current job, I spent six months in Bangalore, India, doing research for a lab that was part of a design school in the northern part of the city. I did not drive while I was in India – I took public transportation and little “autos,” which resemble a golf cart in terms of size and a lawnmower in terms of sound. At that time Uber was barred from India. The driving style in Bangalore struck me as very different from the States: sometimes the traffic lights/stop signs are ignored, sometimes drivers go well beyond oncoming traffic lanes, sometimes when a freeway exit is missed drivers throw their cars into reverse on the freeway. I saw enough daily to get my heart pumping.

I was doing research to understand what the notion of “smart city” might mean in India? As part of the research, I made cold calls to different innovation centers and companies, setting up expert interviews that would inform the research. I learned a lot about how companies had explored the concept of “smartness” in cities. In retrospect, the interview part was easy. Finding the location of the interviews was the challenge.

I had a smart phone. I had a camera. I took photos of the locations on Google maps on my computer or on my phone in case the connection on my phone was lost or hiccuping. One time, I got on the bus headed south and rode it two hours deep into the city to a neighborhood I was unfamiliar with. I hopped off when it seemed like I was close to where I needed to be. There was a queue of auto drivers at the bus stop. I showed my phone and camera screens, with their neat pin-point of my destination on the digital map, to the first driver in the queue. I showed him the address: a building number and street name. The driver waved me in. “No problem!” I thought to myself. I smiled and held on tight to my bag and the rail of the auto. We were off! Turning and bending through little streets and big ones, weaving in between cars and buses. We flew past people crossing the street, animals doing the same, and carts selling food and tea. We drove and drove and drove some more. Minutes led to double-digits. The driver was flying…in what felt like circles. Checking the time, I thought “Oh boy…”

Eventually the driver pulled over to ask other auto drivers for help finding the location. Local folks came to help. A cop or some kind of military person joined in the effort. The mass of people tried to help, pointing around like the scarecrow from the Wizard of Oz guessing all directions to try next. They discussed, pointed, checked and double-checked the address and the maps. At last I got a solid idea: I called my interviewee and he chatted with the driver. We met in a place that the driver could find and then I walked with the interviewee to the building together.

Afterwards, a colleague let me know that the European conventions of maps as we know them don’t make sense to some people in India who have never seen a map in that form. Also, Bangalore is constantly changing, adding streets and changing names of streets. Later on I learned that landmarks are the way to go, as well as calling people sooner rather than later. Still, the worst case scenario was handing my phone to friendly-looking strangers to communicate with a driver when I’m really lost and it worked. A quick shout out to the kind and patient people of Bangalore: Thank you for your constant help getting me to and fro!

Side note: It’s possible my geographical difficulty is just a me thing. More than once I’ve gone to conduct research at the wrong Kroger store on the same street here in Cincinnati!

What Users Do has published our conversation about user research and war stories. Below is a tiny excerpt but please read the whole thing here.

Your book ‘Doorbells, Danger and Dead Batteries’ is a collection of war stories from the world of user research, collected from real life researchers. Are there common themes between the stories?

That’s how the book is structured, actually. Each of the 11 chapters considers a particular theme, something that is of particular challenge to researchers. Some of these are obvious (such as participants we have problems with, or the potential to end up in a dangerous situation) and some are less obvious (encountering not-safe-for-work content in the field, or dealing with our own emotions as well as those of our participants). The chapters begin with an essay by me, include a handful of different stories about the theme, and then wrap up with takeaways that researchers can use to develop their own practice.

You say that improving research skills is about coming to grips with our own ‘flawed humanity’ – how so?

Pulling off a research program is an enormous logistics exercise. Coordinating materials, participants, times, locations, stakeholders, incentives, recording equipment, and on and on. When you work inside a commercial enterprise, there’s a desire to optimize processes, create spreadsheets, build checklists, summarize objectives, deliver key takeaways, etc. But the truth we may forget is that research is an activity that researchers perform with participants. People who make mistakes, people who come to the session with something else going on in their lives, people who have emotions, people who have different verbal abilities, people who have different expectations of the session, and on and on it goes.

While the optimizing efforts are important, these will always be person-to-person interactions. You can fight that and always be frustrated or stressed or disappointed, or you can embrace that as part of the joy of doing research and a source for richer learning. Learning to overcome the pull of frustration and to find a way to actually embrace it is where we can personally grow in our practices.

On the season finale of The Americans, there’s a bit of a post-mortem (so to speak) on a mission, including this bit of dialog

The people back home who aren’t in the field, sometimes they get what we do and sometimes they don’t. But when you’re in the field you have to make split-second decisions. You don’t always have the luxury of thinking things through every time…it’s important to be honest about mistakes. But acknowledging them doesn’t always keep them from happening again.

Obviously they are referencing a different sort of fieldwork. But the lesson applies, nonetheless. For more on this theme, check out my new book Doorbells, Danger, and Dead Batteries: User Research War Stories.

This story was originally published on behalf of The Field Study Handbook.

Last year I was working on a project for a financial technology client. Finding participants is often a challenge, but on this project, for small business owners, it was particularly difficult.

We had hoped to base this research on previous studies, but it proved difficult to glean details about how previous studies were done. There were rumors that another team, elsewhere in the country, had developed a segmentation algorithm, but voicemails and emails went unanswered. We heard about great participants from previous studies that we should revisit, but no one would get back to us. The schedule ticked by and the pressure mounted. In the end, we were left with no choice to work around these limitations. Finally, I began to approach recruiting agencies.

My go-to recruiting team refused to take the assignment on as they had, ironically, recruited for one of these previous studies and felt like they had tapped out the local market. Another company had gone out of business, and a third didn’t think they could accomplish the recruit.

I ended up with a recruiter I had never worked with. In the end, I think they did a good job, but a new relationship added stress to the increasingly complex recruiting process.

In our introductory call, one of our recruits expressed surprise and concern that there would two of us visiting his very small office. We eventually agreed that even though it might be cramped, it would be okay. The recruiting agency, when asked about this disconnect, reassured me that they made it clear, as per my instructions, that there would be two of us. I was confused, as the participant had insisted they had never told him anything about this.

Later that day I got an email from the participant, who sought reassurance about the purpose of the interview. He had clicked on my website (seen in my email signature) and was concerned that I was actually going to be pitching him my services. He had been involved in a focus group through this agency before, and presumed this would be something similar. I confirmed that this was not a sales pitch.

A few days later we met with him in his exceptionally cramped one-person workspace. As the interview unfolded, he abruptly stopped and directly, yet politely expressed confusion and discomfort about the interview itself. Why were we asking these questions? Who do we represent? How are we going to use this information?

It took a long, unhurried conversation about the process and our objectives to put him at ease. We resumed the interview and learned a great deal about his truly amazing businesses, past, present, and future.

—

I emphasize his politeness in stopping the interview, because now, when I go back to the transcript, that’s what I see. But at that time, sitting in that interview, it didn’t feel that way. It felt aggressive and angry and I spent the remainder of the interview feeling uncertain about our rapport. I overcompensated with excessive deference, people-pleasing, and probably flattery. That’s not a comfortable feeling and it’s not conducive to a good interview. I have empathy for someone feeling uncomfortable about something as odd as two strangers with a video camera coming into their office space to ask about their professional history. It’s easy to mischaracterize people that don’t “get it” as difficult. And I assume that I am pretty good at managing expectations at all the common points of failure in establishing rapport.

But boy it’d be nice if we had someone to blame. That guy was a jerkface! The recruiter didn’t do their job (and then lied and insisted they did!). Steve didn’t handle the first call or the interview kickoff properly! Yet it doesn’t seem like any of these are true.

While I felt sheepish at the end of the interview, I was surprised to get a LinkedIn request from the participant immediately afterwards. And, I guess, less surprised when I heard from him a few times weeks later about not receiving his incentive payment (This was one of the very few studies where I asked the agency to send checks after the interview was completed, rather than handing people the incentive directly myself. Mistake? I don’t know). When I followed up with the recruiter about the missing incentive, I heard in some detail how this participant had already called and yelled at the admin staff.

And so it goes.

Last month I spoke at the O’Reilly Design Conference about user research war stories. The video is now online and I’ve also embedded it below.

Here are the slides

Last month I spoke at Interaction 17 about user research war stories. They’ve put the video online and I’ve also embedded it below.

Here are the slides

and a sketchnote by Chris Noessel

The talk features new war stories from special guests Elizabeth Allen and Noël Bankston.

Update: Here’s the text of my talk, in Spanish.

It’s been two months since Doorbells, Danger, and Dead Batteries was released. Here’s links to presentations, podcasts and more stuff related to the book.

The Book

• Buy from Rosenfeld Media
• Buy from Amazon
• Foreword by Allan Chochinov
• Excerpt from chapter 3
• Excerpt from chapter 11
• FAQ
• Testimonials
• Images

Reviews

• Goodreads
• Amazon

Podcasts

• UX Podcast
• Rosenfeld Media
• User Experience Podcast

Interviews

• O’Reilly Design
• UX Mastery AMA
• Rosenfeld Media

Presentations

• Amuse UX (video)

Other

• War Stories archive
• New story by Noël Bankston, told at Interaction 17
• New story by Elizabeth Allen, told at Interaction 17
• Nick Bowmast on war stories
• Susan Dray on war stories

Noël Bankston is a UX Research Lead and Human Factors Engineer at Zebra Technologies, currently living in Queens, NY. She told this story live at the Interaction 17 conference.

“So Jim, what would you like to do for lunch? “My treat!” It was the moment I had been dreading all day, ironic since I am a lover of food. I was trying to sound chipper but I was worn through.

It was 2 pm and I was starving. I was sitting in the cab of a 48’ tractor trailer in Lowell, Arkansas. This was my first “ride along” research trip and I had not come prepared with snacks. I was doing in-depth generative research of the pick-up and delivery process for a freight company and hadn’t known that we don’t have lunch until all the deliveries were completed.

I was also not prepared for the weather as I am from up north and I thought the South would be hot in late May. It wasn’t – it was a constant drizzle and cold. So I was sitting in the cab feeling small and tired in the oversized loaner jacket that the dispatcher had given me. We had been on the road since 8:45 am but I had arrived at the trailer dispatch site even earlier to observe the set-up process. And that should have been fine, because on a normal day, Jim finishes around noon. But today we saw all the exceptions – an unprepared customer, incorrect paperwork, an obstructed delivery dock, and poor routing. As a researcher, it was a gold-mine as I observed where problems occurred and how Jim handled them. But as someone who is mildly hypoglycemic, it meant I was getting hangry. It had been a long morning of climbing into and out of that cab, learning which hand to place where to get the right leverage to pull yourself up as you step onto the step that is only wide enough for half your foot. And I don’t know how many of you have ridden inside of a tractor trailer but it is loud and you feel every bump.

In that moment as I asked about lunch, damp, tired, and hungry, I thought back on the the anxiety I had felt earlier in the day about lunch. A co-worker told me that on his previous ride-along they had eaten a burger from a gas station mini-mart. Even on a normal day that would make me uneasy, as gas stations aren’t known for freshness and hygiene. I knew that this type of research means being available for wherever the subject takes you, but I was really hoping that didn’t include food poisoning.

But at this point, 8 hours from my previous meal and having no idea what part of town we were in, who was I to be picky?

“So Jim, what would you like to do for lunch?”

“I just want a salad. I try to eat healthy.” I gave a huge sigh of relief, accompanied by a rumble of rejoicing from my stomach. It seemed that between the two of us, I would be eating the bigger meal. I found a nearby Mexican restaurant on Yelp. While enjoying the flavor combination of fresh cilantro and lime with nary a fryolator in sight, I realized how I had been making assumptions about “truckers” based on stereotypes rather than letting the research reveal the truth. And those assumptions were also judgments about health and lifestyle. Jim was aware of the health effects of his job and wasn’t going to turn down an opportunity to have a healthful meal, especially when a researcher was paying! One of the reasons truckers eat unhealthy food is both cost and convenience. Truck stops get food fast and are less expensive. Unfortunately, our food system is set up in a way that fresh, whole food costs much more than highly processed, industrially produced food.

I won’t be able to eliminate all my biases or preconceived notions but I can grow in my awareness of them. I have been on many more ride-alongs and other types of research trips since then. You better believe I always have a granola bar with me.