Snipped from an article today about chemical-based cleaners in the home

Cory McKee, 27, a stay-at-home mom of three in Tridell, Utah, started ordering Seventh Generation brand cleaning products online two years ago after learning that her oldest child, now 7, had celiac disease, a gluten intolerance. Ms. McKee said that although the disease is not caused by toxins in the home, dealing with it raised her awareness about other health issues. [italics mine]

“That really woke me up,” Ms. McKee said. “I really need to make sure our home is safe.” She lost confidence in the cleaners she had been using in part because the labels of some products do not list all of their ingredients. That made it impossible to know what her family was being exposed to when she sprayed the windows, she said.

I love this, even after it’s gone through the journalistic clarifying filter. People’s ideas jump from one arena to the next. We conflate different concerns. Ask someone about their eating habits, and they’ll talk about exercise, or ask about being fit and they’ll talk about bedtime, or how to stay calm and deal with stress. We’re not good at compartmentalizing. And so ethnographers, using a conversational tool, encounter this all the time. A decision about one thing is related to a concern about something else. Even though there was no causal relationship between the celiac disease and the cleaners (and not to mention that they child already has celiac disease) the person being interviewed puts forth a causal relationship.

I hear this sort of thing all the time when I talk with people, and it’s usually much less clear than this. Complex purchase decisions, tasks, and lifestyle choices (of which our lives are full) often are told in slippery stories that start off in one place and end up in another. Teasing those apart (as this writer did here), asking for clarification, and being open to understanding how A could possibly connect to B in someone’s mind are crucial to getting at those applicable insights.

Interesting methodology (and findings) for understanding the factors that influence our eating behaviors at the Food and Brand Lab at Cornell

In an eight-seat lab designed to look like a cozy kitchen, Dr. Wansink offers free lunches in exchange for hard data. He opened the lab at Cornell in April, after he moved it from the University of Illinois at Urbana-Champaign, where he spent eight years conducting experiments in cafeterias, grocery stores and movie theaters.
…
Although people think they make 15 food decisions a day on average, his research shows the number is well over 200. Some are obvious, some are subtle. The bigger the plate, the larger the spoon, the deeper the bag, the more we eat. But sometimes we decide how much to eat based on how much the person next to us is eating, sometimes moderating our intake by more than 20 percent up or down to match our dining companion.
…
Moviegoers in a Chicago suburb were given free stale popcorn, some in medium-size buckets, some in large buckets. What was left in the buckets was weighed at the end of the movie. The people with larger buckets ate 53 percent more than people with smaller buckets. And people didn’t eat the popcorn because they liked it, he said. They were driven by hidden persuaders: the distraction of the movie, the sound of other people eating popcorn and the Pavlovian popcorn trigger that is activated when we step into a movie theater.

Dr. Wansink is particularly proud of his bottomless soup bowl, which he and some undergraduates devised with insulated tubing, plastic dinnerware and a pot of hot tomato soup rigged to keep the bowl about half full. The idea was to test which would make people stop eating: visual cues, or a feeling of fullness.

People using normal soup bowls ate about nine ounces. The typical bottomless soup bowl diner ate 15 ounces. Some of those ate more than a quart, and didn’t stop until the 20-minute experiment was over. When asked to estimate how many calories they had consumed, both groups thought they had eaten about the same amount, and 113 fewer calories on average than they actually had.

A sidebar listing other experiments and results is here (link may expire).

Molly Steenson recently gave an excellent presentation at Giant Ant, describing her work in Bangalore with Microsoft Research. My notes are at Core77.

Here is a link to a podcast page for my recent talk at Adaptive Path’s UXWeek.

Effective user research requires both observation and interviewing. When doing research we strive to get outside our own default expectations and perceptions, in order to better see the details of what we’re looking at, in other words, to understand the cultural context. This third component is the most crucial to innovation. Interesting things happen when we leave our homes and our comfort zone, perhaps in another country where business, language, food, and more is beyond our own frames of reference.

Steve Portigal, founder of Portigal Consulting, offers expert tips in both observation and interviewing, and considers the challenges and opportunities in conducting research abroad. He believes that one way to better understand a different culture is to look at how things in your own culture are handled differently. He gives some examples of how some things are promoted differently in Japan than in the United States. He states that mundane observations reveal important cultural differences.

Here are the slides

I’ll be curious how others fare; I couldn’t get it to go past slide 6, although I could go to the end and page backwards. I’ve reported a bug and hope it gets fixed, but since it’s just been launched and getting some buzz, who knows. I re-uploaded and replaced it with a version that works. They are working on tracking down the problem with the last version.

I have been running some in-home discussion groups for a design project recently. The ebb and flow of context is just so interesting to me and highlights the challenges of getting “all” the information.

[none of this is verbatim]

Q: If you and your wife own one iPod, how do you determine who is going to use it?
A: Well, for commuting, it’s either the iPod, or the New Yorker.

Two scenarios are likely:
1. She takes the iPod on the train and he drives their lovely car, a New Yorker.
2. Whoever takes the iPod gets music, and the spouse gets to read the most recent issue of the New Yorker magazine.

There’s always the clarification question: “When you say ‘New Yorker’ are you referring to the car or the magazine?” but in this case, we didn’t get to ask that, and I was confused at the time. It’s clearer looking at the video that they are talking about the magazine.

Later on, the same guy (still talking about iPods) tells us “Well, when you do that, it looks more Zen. It actually looks like the competition.” and moments later another participant adds “Yes, it’s like he says, very Zen, very Japanese, very spiritual.”

But that’s not at all what he meant. He meant another type of MP3 player, the Creative Zen.

I understood what he meant, at the moment, but the other person, didn’t. And it wasn’t possible in the flow of things to clarify (and maybe not even necessary).

I have powerful memories of being in Mr. Collison’s grade 6 class, and seeing him do this sort of thing all the time – missing a word or a piece of context of what someone said and riffing on it, in entirely the wrong direction. It really made me squirm in my seat to see all this miscommunication around me and have to keep quiet, or at best, wait to offer my insight. I wonder if other people notice this stuff and are as stimulated/aggravated/curious as I am?

I’m totally psyched to report that starting in September, I’ll be teaching Design Research Methods again at CCA in San Francisco, in partnership with Todd Wilkens.

Notes from my UXWeek session are online (not sure how helpful they are without hearing me talk through the issues, (Update: you can hear me talk through the issues here) but if you want to, check out Dan Saffer’s notes or the notes from the wiki.

Hawai’i makai/mauka signs
different orientation toward navigation: toward mountain, toward ocean
difference in how we move through space

so what?
mundane observations reveal differences in cultural needs or drivers

Refreshing piece in the NYT (since it omits the usual players and the jokes about anthropology) about the importance of getting out of the office and getting to where your customers are.

Once a year, though, he organizes a different kind of hunt – which he calls a “branch hunt.” In it, the entire organization turns its attention from the suite to the street – and, by scrutinizing the fine details of how banks interact with their customers, sees the market from a new perspective.

“The most thoughtful and articulate strategies tend to come from the big banks,” Mr. Brown explained. “But their actual results seldom bear that out. When you walk the streets and look at what’s happening, the gap between strategy and execution becomes obvious. We can’t just listen to what executives say. We have to see with our own eyes what customers are experiencing.”

The dress code for a branch hunt is casual, but the approach is rigorous. For its fourth annual hunt, Second Curve pinpointed the location of every branch of every bank on the East Side of Manhattan, from 25th to 86th Streets.

All the firm’s employees – the analysts, the compliance officer, the computer geek, the receptionist – divided into teams, were assigned specific avenues and streets and set out with digital cameras, audio recorders and four crisp $100 bills for each team. They spent time at the branches, chatted up bank employees, opened checking accounts with the company-issued cash, snapped photographs – not a popular practice with bank security – and captured the flesh-and-blood experience of being a customer.

After the hunt, the teams returned to headquarters and described what they saw, from stories about horrible or remarkable service, to reports on flat-screen televisions that were meant for customers’ viewing but were occasionally found in truly bizarre places where the public could not see them.

Todd writes, over at the adaptive path blog

Focusing exclusively on tasks and goals means that you tend to ignore or de-emphasize all of the activities that people engage in that are specifically not goal-oriented. It also means that you will often ignore the messy jumble of activities that take place around but are not oriented toward your system. This is not always problematic but it quickly becomes so when you are designing for multiple contexts and mediums. When it comes to designing for the total experience, the activities that have little to do with the system you are designing are often just as important as those that are central to it. More than ever before, people switch from one context to another rapidly and often. They were in the outskirts of Cleveland mowing their lawn then the cell phone rang and suddenly they’re planning a trip to Thailand.

The thesis of the piece, as I read it, is not simply to shift methodologies (do ethnography and forget usability) but to change the fundamental way that we structure and act on the information we gather about the people we are designing for.

It’s a great challenge for organizations, and for consultants, because there’s powerful cultural infrastructure that drives what is an acceptable piece of new knowledge, and of course, what isn’t. In order to see how you might act on something – what do we DO with this information – requires a shift in perspective. And those don’t happen overnight, when they do happen.

Here’s part 2 of my conversation with LukeW. Please see Functioning Form for part 1.

Luke Wroblewski

Let me try to clarify by taking a step back. I see field research as a way to remove some of the tarnish that comes with more “traditional” market research like focus groups and surveys. The common perspective is that people in a focus group or survey won’t really tell you what they want or how they do things because often times -they can’t. They are a level removed from the actual activity and as a result may leave out key details or considerations they use the make decisions.

The classic example, and I can’t recall where I first read it, is the washing machine manufacturer that polls thousands of potential customers and asks them “what features do you want in a washing machine?” The responses they get back are: “just the basics”; “i just want a simple setting for colors and whites”; “nothing too fancy”; etc. So the company makes a bunch of no-frills, feature-lite machines and they don’t sell because when it actually comes time to buy a machine the same people that said they want “simple above all else” fall prey to feature-sheen. “Oooh but this one has more features…” I’m sure you’ve heard a similar tale or two.

So what we have here is people saying they do one thing then going out and doing something totally different. Field research should ideally be there at the point of the sale -in context- to enable the company to see what really happens.

Now let’s go back to my original question about digital context. In all the methods you described above -great list by the way!- we’re asking people to tell us what they’re doing rather than being there -in context- when they are doing it.

Maybe I’m picking nits here but I know there are lots of “hidden” subtleties within digital social systems that govern how people behave. There are contexts of when and where that alter behavior. As an example, during a home visit a buyer on eBay may tell you: “I leave positive feedback when I get an item in good condition.” Their actual behavior, however, differs. They may or may not leave feedback based on the type of seller (professional or amateur), how much feedback they have, how much feedback the seller has, the category they are buying from, their intentions for the item after they get it (resell, return), and so on.

I guess when I think of people that spend hours every day immersed in something like World of Warcraft I feel there’s more to their behavior and motivations in that digital space than they can explain in words. How can we be a fly on the wall within that digital context? Or is what I’m looking for already covered by the methods you outlined?

Steve Portigal

Any situation where you have someone telling you about their own behavior is going to include some amount of bias (and let’s pretend for the sake of discussion that our own bias isn’t an additional factor). In focus groups, those influences are hard to leverage (complex peer dynamics, sterile environment, closed-ended discussion), but in contextual research, we can try to take advantage of showing and telling, for instance. Having someone walk through their previous feedback log, and explain, is illustrative of patterns that person may not explicitly be telling us.

Q: Are you leaving feedback for the seller?

A: I leave positive feedback; it’s really important, I usually will look at the condition and decide based on that.

Q: [points to computer screen]Can you walk us through some examples of feedback?

A: Sure, umm, here’s one I left last week. The item was in pretty good condition, so, well, I only left 2 stars, because I didn’t get it right away. And this one here, I left 4 stars because the last time I bought from them it was great. Yeah.

Q: What’s that icon over there?

A: Oh, there’s a bunch of items awaiting feedback.

Q: How often does that happen?

A: Well, I’ve got about 35 in there, some of them let’s see oh yeah, some of them go back about 3 months, I guess.

Now, I’ve totally made that up to support my point so let’s not treat it as data, but as a likely scenario for a dialog in an interview. So much of the process involves triangulation – asking the question at different points in the interview, getting demonstrations as well as declarative statements as well as stories. When you come out of the session, you have to ask yourselves what you think that person’s approach to feedback is. And it lies somewhere in between, but it’s ultimately an interpretive answer.

We can measure people’s TV viewing habits, let’s say, with a Nielsen box, and we can ask them about their behavior. And history shows (like the washing machine example) a disconnect. People under-report their TV watching. It’s easy to think about why; TV is bad, it’s better to show yourself as someone who reads books and goes on hikes than it is as someone who watches a ton of TV. The insight comes out around the delta between the observed and the reported. Of course, not every gulf is an insight, or one that you can use.

And maybe to make it more simple, it’s easier to talk to people about what they did in the past, and why (i.e., leaving feedback) rather than their overall attitude. People make generalized statements but the actual examples contain a lot more subtlety. This of course leads to one reason that companies like to reject any form of customer research – because people can’t talk about what they will do or what they will want. My short answer to that is that the researcher is the one doing the interpretation; in all of this we aren’t simply collecting responses verbatim – we are dynamically choosing different questions and making inferences in order to build our own model of how people will behave.

That said, I had an interesting conversation the other night with Zachary Jean Paradis, a student at the Institute of Design. He described how he had tried to do an ethnography of World of Warcraft (a MMORPG), where presumably a lot more of the behavior to be understood is taking place inside a virtual world. He felt like his traditional tools of ethnographic research didn’t hold up, and he was wishing for another month to better refine his methods. Gaming is an interesting example (and one where I don’t have a great deal of personal experience) of the online-behavior-studied-offline that we’re talking about. I’ve heard that some researchers will videotape the faces and body language of people while they are playing; I imagine you could play those back along with the matching gameplay and have people reflect on what they think was happening at the time. You can see that technique in Gimme Shelter, where Mick Jagger is watching the footage from Altamont while they interview him. A UK company called Everyday Lives does this sort of user research exclusively, preferring to passively observe, and then only interview when there’s a video record of the event to be discussed. I think it’s an interesting tool, but I think we need an ever-expanding palette of methods to deal with new situations as they emerge, rather than dogmatically rely on a single approach.
Am I still dodging your question? Or are we any closer?

Luke Wroblewski

Maybe I’m dodging your answer! One thing you said, though, really resonated with me “getting demonstrations as well as declarative statements as well as stories.” Since we can’t actually be a fly on the wall within complex digital systems yet -and I say that because the tracking software and log analytics software I’ve used is still a ways off from being nuanced and effective enough to match what we can do it the real world- that’s how we need to understand context: through demonstrations, declarations, stories, and of course observation of what people are actually doing on screen. Personally, I do think as digital environments become even more immersive and complex, we’ll need additional methods.

That said, let’s jump into the other topic I wanted to bring up with you. Without getting into pure semantics, why do you think a lot of ethnographic or field research is being characterized as “design research”? Is it user experience design teams within large companies trying to own the research process/data? Is it an attempt to differentiate the type of customer insights a human-centric problem solving approach can uncover from the types of insights Marketing departments have traditionally owned -like customer segmentation? Or does this type of research intrinsically belong in the “design world”?

Steve Portigal

I agree that there’s more to behavior than can be explained in words. It’s up to us to look for the deeper meanings between the words – what is said, what isn’t said, and how it’s said. As far terminology goes, I agree with your suggestion that the label can be an attempt to distinguish the methodology and/or the results from market research, and the departments that do market research. I’m so frustrated by the chaos around methodological labels. I’m sure within organizations they can create a locally-relevant nomenclature (they can, I’m not sure they do) but once you leave the boundaries of their company (through any industry discussion, conference, or online group) they end up sowing confusion. The vendors, of course, who move between companies are even more guilty. There’s a desire to differentiate from the other providers by claiming some proprietary take on doing research: Context-Based Research, PhotoEthnography, Rapid Ethnography, etc. (some of those may be actual methods claimed by actual firms; others may just be me riffing). It’s tough to balance exploring the ideas and staying “on-message” isn’t it? I guess that’s why I don’t take kindly to the terminology wars; they seem to make it more confusing for people.

Luke Wroblewski

So this time you did dodge my question! I’ve consistently heard “design” added to “research” when describing the type of activities we’ve been discussing. Any thoughts on the inclusion of the design label? I know you find yourself in lots of designer-focused events like Design 2.0 in San Francisco, Core77, Overlap… is there really that strong a connection between design and ethnographic research? Why doesn’t this type of research feed business models more than mock-ups? From my experience, the designers eat this kind of data up, the business folks are slow to act on it. What’s you take on that?

Steve Portigal

One suggestion is that the term is historical. Bringing the tools of ethnographic research into product development was led by a few firms that were self-described design firms (like my old company, GVO) or that had ties to design (Doblin, with their connection to the Institute of Design). I would also say (and this is a gross generalization) that market research tends to focus on the evaluative and design research tends to focus on the generative. That’s more about the goals of the research sponsors than anything inherent in the methodologies (since ethnography is ethnography).

And I think those goals or orientations that differ by discipline will affect the gusto with which they eat this stuff up. In many cases, designers are faced with tangible goals. They are committed to acting, since the product has to be designed, and is going to launch. The information they gain from research can help them solve a near-term problem (i.e., what is the organizational framework for a navigation through a space?). Even though my strategic recommendations can be as tangible as my tactical ones, they ask for actions that are much slower (i.e., launch a newsletter that addresses the transparency concerns of customers), more tangled up in organizational (same example) and resource issues (same example), and with many degrees of freedom to create a good solution (and as non-designers, that can be paralyzing).

Luke Wroblewski

Thanks Steve!