Posts tagged “participants”

Facing Your Fears: Approaching People For Research

intercept

I’ve always found intercepts – where researchers stop people on the street and ask them to participate in a quick study – to be challenging. (I also prefer to have longer interactions with people and even have them prepare for those research conversations, but that is a bit outside the point here). In Facing Your Fears: Approaching People For Research Carolyn Chandler breaks that challenge down into many pieces and addresses each of them. It’s a wonderful article because it gets deeply into the specifics and considers the mindset we bring to the activity and how to reframe that, in many different ways.

Rejection is people’s number one fear when approaching strangers. Hearing no has always been difficult, whether it’s a polite no or an angry no followed by a rant. Either way, it stings. Your response to that sting, though, is what matters. How do you explain the rejection to yourself, and does your explanation help or hurt you?

Martin Seligman, one of the originators of positive psychology, conducted a study in the ’70s that gives insight into the types of mindsets that make people feel helpless. Seligman found that those who exhibit long-term “learned helplessness” tend to view negative events as being personal, pervasive and permanent. In other words, if a person is rejected, they might rationalize that the rejection is a result of their own failing, that everyone else is likely to reject them as well, and that they can do nothing to lessen the likelihood of rejection.

When you prepare to approach someone, consider instead that, if they say no, they aren’t really rejecting you, but rather rejecting your request. It’s not personal. Maybe they’re in the middle of something, or maybe they’re just not in the mood to talk. The rejection is fleeting, and the next person might be perfectly happy to participate.

One way to recruit in-home research participants

Credit card issuer Capital One isn’t shy about getting into customers’ faces. The company recently sent a contract update to cardholders that makes clear it can drop by any time it pleases. The update specifies that “we may contact you in any manner we choose” and that such contacts can include calls, emails, texts, faxes or a “personal visit.”

Yep, that’s how you get willing research participants – add it to the Terms of Service! Sure, I’m kidding; that’s not really what this article is about (it’s about the credit-card company claiming rights to repossession for non-payment). Still, this ups the creepy ante for visiting customers and makes the trust aspect of recruiting participants just that much harder.

Full story

Cordy’s War Story: A Crisis of Credibility

Cordy Swope (Twitter) describes himself as either a Design Researcher with Grey Hair or a design researcher with grey hair.

IDEO. NYC. Early 2010.

I had been summoned from Europe to lead a project about the future of education in the US. At IDEO, there is a well-established a code of ethics for site visits. This code takes extra measures to protect the privacy of informants – especially their identities and contact data. IDEO also has sensible, street-smart guidelines for fieldwork in sketchy environments. In previous jobs, I had seen a situation in which two of my female design researchers had to go to remote, sparsely populated parts of the Midwest and visit big, burly, smiling men who stored every conceivable power tool in their dungeon-like tornado cellars.

There is never a shortage of people in NYC though, and recruiting there offers many delights. For instance, NYC is one of only several places where it is possible to recruit for impossibly specific profiles like: “Seeking 3 single dads who have volunteered with their children at a local charity organization within the past 2 weeks, and who also must struggle with their own gender identity and make at least $150K/year.” In the Tristate, if you are one in a million, by definition there are at least 22 of you.

Our recruiter used Craigslist for most projects and straightaway found us one of our targets: a working mother who had successfully completed a BA online while still raising a family. I had a new team and my associate design researcher was an eager, empathic and articulate ethnographer doing her first project at IDEO. We headed out to Inwood in Brooklyn for our first site visit, hoping to get insights from this working, baccalaureate mom.

During the ride, I played the senior mentor guy, offering advice about doing ethnography “in a design context.” We arrived at the address in Inwood, an obscure part of Brooklyn that looks like a sad, dilapidated part of Queens that in turn, tries to look like a nondescript suburb in Long Island. We were buzzed into the building, walked up to a door and were greeted by a large woman with a curly red mane of hair. Her name was “Roberta-but-call-me-Bert.”

She let us in. The apartment was dim. It smelled of litter box mixed with burnt Dinty Moore beef stew that Ramon, Roberta-call-me-Bert’s husband had overheated on the stove. The dingy plaster walls were covered in old shopping lists, written in a mangled scrawl that suggested vaguely menacing pathologies and personality disorders suffered by their author.

The sofa we sat on smelled of cat piss, and the living room offered up no pretense of ever having been cleaned. We sat up straight, made eye contact in that standard, pious, non-judgmental manner that earnest ethnographers often adopt. We began the paperwork. We were offered water and politely declined.

I asked her about work, family, free time; all of the perfunctory questions before we got into her BA experience. Since I was the seasoned professional, I led the discussion, “Tell me a story about your favorite class…”, “Did you make friends with your classmates?”, “Do you still keep in touch?” Since my associate was taking notes, I focused on keeping the discussion moving and letting Roberta-call-me-Bert lead us to all sorts of exciting insights.

The trouble was, she didn’t.

“Oh, I don’t remember much about that class,” she said about her favorite statistics course she took just before graduating 18 months ago. “Yeah, I pretty much kept to myself, because I had to work and raise a family, you know?” I nodded my head earnestly.

I began asking her questions about change: “Do you view your daughter’s education differently now since you got the degree?” “Not really,” she said, as her daughter ate ice cream from a container while watching a YouTube video about dog fighting.

We eventually went on our way. Once out the door, I was about to launch into the debrief. Since I was the experienced one, I was going to teach my associate a simple, time-honored 20 minute structure I often use for debriefs: Interesting Behaviors/Motivations and Drivers/Problems and Frustrations/Opportunities.

I noticed that she was grimacing.

“What’s wrong?” I asked.

“That was a waste.” she replied.

“What do you mean?” I asked.

“She lied, she never went to college.”

I was gobsmacked.

And she was absolutely right.

There were no interesting behaviors. There were no drivers or motivations. There were no problems or frustrations. There were no opportunities.

There was no diploma. It was “packed away somewhere.”

We returned to the office. Another colleague was leading a project in men’s fashion and desperately trying to recruit shop-along dyads of couples in their 40’s and 50’s where wives selected the husband’s clothes. She said they had already recruited one couple on Craigslist and that her name was Roberta from Inwood, Brooklyn.

Sean’s War Story: Pockets full of cash

Sean Ryan, a corporate ethnographer, reflects on a fieldwork experience where he learned first-hand some crucial lessons when going into another country: pre-recruit participants, and do some basic homework about where you are going.

It was back in my early days as an ethnographer. I was still a young pup in the field, doing consulting projects. I was teamed up with an Elder Anthropologist – a Puerto Rican woman who lived in Guadalajara, named Luz. We were doing a project for a major pharma company who had just had great success with a new oral care product, so they thought they would try an ethnographic exploration to uncover any other unmet needs. I think their aspirations at the time were something like “We want the next $500 million consumer product!” Luz and I were to visit two field sites in Mexico: Guadalajara and Tijuana. Living in Los Angeles, I was relatively close to the border and it wasn’t yet seen as that dangerous to go to Tijuana (i.e., no Mexican mafia drug lord street battles…at least you didn’t read about them in the papers everyday). But I still had my reservations, not possessing any potent Spanish language skills (outside of the slang I had picked up from bartending in a Mexican restaurant in Long Beach).

Having only been to Tijuana once to explore the finer points of Avenida Revolución (read: drinking tequila shots with college kids and having my head shaken back and forth by a woman with a whistle), I had no real frame of reference for doing fieldwork in TJ. As I quickly learned, neither did my counterpart Luz. She had some relatives in TJ, but had never done fieldwork there. And so we made what we later realized was a critical error in not pre-recruiting participants before we went into the field. Upon arriving in Tijuana we quickly found ourselves literally approaching people in the streets, in shops, etc. to ask about their oral care routines (a strange encounter for the locals I’m sure). While this has all the hallmarks of classic guerrilla recruiting it’s never a comfortable situation to be in, especially in a foreign country. Luz was doing her best to recruit people while I stood by idly awaiting our field day fate.

Eventually we started to have some success…or so we thought. One woman who worked in a nice department store in downtown TJ offered to let us come to her home after work. We got her contact info and told her we would see her that evening. We were offering $150 in US cash (this was more than 10 years ago) to interview the participant and observe their oral care routines. This, no doubt, was more than substantial for an incentive. So we were quite confident that we would have no problems grabbing participants on the fly. That evening, we made our way to this woman’s house via an old Crown Victoria station wagon taxi (with the suicide seats facing out in the back). Once we got to her town we approached the participant’s door and gave it a confident knock…but nobody answered. We waited a few minutes longer and knocked again…still no answer. This was before mobile phones, so we couldn’t exactly call this woman on her cell. We sat and waited for 15 minutes, but then realized that our day was quickly wasted on a participant who, for whatever reason, decided she did not want to do the study (Perhaps she thought the $150 was too good to be true?). In a moment of desperation, Luz decided to frantically go door-to-door in this small community, hoping for a shot at someone’s teeth and mouth. But to no avail.

This disastrous field trip continued. The next day we tempted fate again by preying on another unsuspecting citizen of Ciudad Tijuana. Once again, we arranged to go visit a shopkeeper’s home later in the evening. Once again, we had no idea where exactly our little field visit would take us. And once again we crammed ourselves into an old Crown Victoria station wagon. This time we were left off at what appeared to be a small village of Gypsies. It was, in fact, just a typical working class abode on the outskirts of the city. I brazenly brought out my Sony DV camera with the Carl Zeiss lens and began filming the local scene as we walked through the streets to find the right home. We were very excited to actually find the participant’s home and then to actually find her in her home!

It was a very interesting interview: the participant was a mother of two, a 9-year-old daughter and a 6-year-old son. We observed their oral care routines, which consisted of going out to the backyard to gather water from a large plastic drum (as there was no running water), after which the children vigorously brushed their teeth with your standard run-of-the-mill Colgate toothpaste and toothbrush. When we paid the mother $150 (US) cash at the end of this encounter, her eyes lit up. I realized at that moment that this was probably more money than they made in a month. And so we broke another field rule: understand your surroundings and pay participants appropriately based on the context. But there was bit of a feel-good moment here too; the client could clearly afford the incentive money, so it was no skin off of their backs.

After this first round of field visits in Tijuana we came back about a month later for a second round, with different participants. We interviewed a relative of Luz’s who lived in a canyon high above where we had visited last time. He laughed out loud when we told him that we had been down in that village only a month ago. He said with all seriousness “Don’t you know that is the most dangerous area in all of Tijuana?!” Of course we had not known this. I thought back to the $800 Sony camera that I slung around in the streets of that village. And then I thought of my pockets full of cold, hard US dollars. I laughed to myself, but thought “I need to be a little more careful in the future if I’m going to make a career of this ethnography business!”

Julie’s presentation on “Finding the Right People”

I recently had the privilege of guest-lecturing for an undergraduate class at the Center for Design Research at the University of Kansas taught by Julia Eschman and Tamara Christensen. The topic was the importance of recruiting the right people as participants to drive powerful insights for design research. I addressed the strategic reasons why, as well as tactics for doing so.

Recruiting is a key component of the design research process. It is to the detriment of project outcomes when it doesn’t get the attention it deserves. I know, I know… it’s a lot more fun to talk about sexy new methodologies and while away our days ideating, but before any of this can happen teams need to think hard about the source of the inspiring data and stories: people.

The presentation is below:

This was the first time I’ve presented this material, so take a look and please share some of your key experiences, snafus, complaints, challenges and learnings and etc. as you find the right people for your projects.

Reading Ahead: Managing recruiting

Reading ahead logo with space above

There’s always something new in every project. Often we encounter a bit of process that we may not know how to best manage it. So we’ll make our best plan and see what happens. We learn as we go and ultimately have a better way for dealing with it next time.

In a regular client project, we write a screener and work with a recruiting company who finds potential research participants, screens them, and schedules them. Every day they email us an updated spreadsheet (or as they call it “grid”) with responses to screener questions, scheduled times, locations, and contact info. It still ends up requiring a significant amount of project management effort on our end, because questions will arise, schedules will shift, people will cancel, client travel must be arranged, etc. etc.

For Reading Ahead, we did all of the recruiting ourselves. Although we’ve done this before, this may be the first time since the rise of social media: we put the word out on LinkedIn, Twitter, Facebook, email to friends, and here on All This ChittahChattah.

While Dan lead the effort, we both used our own networks, and so we got responses in a number of channels, sent to either or both of us, including:

  • @ replies on Twitter
  • direct messages on Twitter
  • Comments on Facebook posts
  • Messages on Facebook
  • Emails (directly to either of us, or forwarded from friends, and friends-of-friends

facebook
A private dialog on Facebook

facebook2
Comments on a Facebook status update. Note that Dan is able to jump in and make contact directly

twitter
Direct Messages in Twitter

Some people were potential participants, some were referrers to other potential participants, and some were both. And given the range of platforms we were using, with their associated restrictions (and unclear social protocols), we had to scramble to figure out who could and should communicate with who to follow up and get to the point where we could see if the people in question were right for the study. We didn’t expect this to happen, and eventually Dan’s inbox and/or his Word document were no longer efficient, and as some participants were scheduled or in negotiation to be scheduled, he ended up with this schedule cum worksheet:

schedule

Being split across the two of us and these different media, eventually we were interacting with people for whom we had to check our notes to trace back how we had connected to them, which was great for our sample, since it meant we weren’t seeing a group of people we already knew.

It was further complicated when we had finished our fieldwork and wanted to go back to everyone who offered help close the loop with them, thanking them for help. Technically, and protocol-wise, it took some work (who are the people we need to follow up with? Who follows up with them? What media do they use), basically going through each instance one-by one.

twitter2

We haven’t figured out what we’ll do next time; we won’t forget the challenges we’ve had but there’s just not time or need right now to plan for the future. If I had to guess, I’d imagine a Google Spreadsheet that includes where we got people from, who owns the contact, whether they are participant-candidate or referrer, etc.). Despite being very pessimistic about the demands of recruiting, we still underestimated the time and complexity required for this project.

Little lies by focus groupies are costly

Nice article about people that lie in order to qualify for market research studies

Researchers call these truth-stretchers focus groupies, a sneaky cadre that adopt multiple identities in order to secure paid seats on the dozens of focus groups that meet every week in the Bay Area.

Firms pay $50 to $100 cash for an hour or two of work that usually involves a moderated discussion about a new product or service with up to a dozen people gathered in a room equipped with a two-way mirror.

The allure of easy money leads hundreds of people every year to treat focus groups as a source of nearly work-free income. Get-rich-quick schemers even advertise focus groups as a source of cash.

And if it means telling a few lies along the way about your favorite brand of frozen pizza or the number of times you have already participated in a focus group, well, it’s no crime to fib to a marketing company.

Researchers go to great lengths to weed out groupies, including the use of exhaustive database cross-checks to ferret out the ‘cheaters’ and ‘repeaters,’ along with detailed screening interviews. Competing firms even share groupies’ names in the reverse form of a ‘do not call’ list.

‘It’s bad for the whole industry so we cooperate with each other,’ said Nichols Research Group Vice President Jane Rosen, whose Bay Area firm purges several hundred groupies a year from its database.

How far will people go?

They sign up with aliases, usually derivatives of their real names with different initials and middle names, Rosen said.

They may use a post office box address under one application and then a home address for the second response.

‘We had a woman sign up for two focus groups on the same day and after she finished the first session, she went out to her car and changed into a new set of clothes and put on a wig,’ Rosen said. ‘Fortunately, one of our people thought something looked wrong about her.’

Q&A Research in Walnut Creek recently foiled a woman who claimed to own a particular brand of luxury car, but the name on the automobile registration she provided did not match her own.

‘We had another man who used his first name for one group, then his middle name for a second group the next day and then a third one the following week,’ said Eric Tavizon, Q&A’s focus group project manager. ‘One of the clients caught him because he mistakenly signed up for events by the same sponsor and they recognized him.’

Of course I’ve encountered this on a much smaller scale; so much of what I do is predicated on a basic foundation of trust (and trust goes in two directions, of course) and it’s lurid and disturbing to consider how that trust can be violated (when do we read the piece about the rapist who posed as an ethnographer to gain access? yikes).

I’ve started a discussion thread on Discovery about this; we’ll see if anything develops.

Series

About Steve