I’ve mentioned Maximum City: Bombay Lost and Found by Suketu Mehta in a previous post, but thought it was worth its own post now that I’ve finally finished it.

We learned about this book on our 2006 trip to India, but it took me over a year to finally get to it.

The book is Suketu Mehta’s collection of stories from his return to India after 21 years. He’s an insider and an outsider all at once. He shares his own experiences (say, in trying to rent an apartment, or get his kids into a decent school) but also picks a number of different subcultures (life in the slums, commuting, gangsters, Bollywood, sex workers, homeless artists, religion, politics, law enforcement) and goes deep. He develops intense relationships over time and tells the stories of the characters he encounters, many of whom live outside the norms that most of us could tolerate. He goes deep enough that as a writer, he’s pulled into writing a screenplay for a Bollywood film.

Although he goes into these subcultures as individual forays, many of the threads overlap (Bollywood and gangsters, the police and politics and religion, etc. etc.) and collide and so a more complete portrait begins to emerge.

I really appreciated having my own experiences contextualized by the author’s similar (if much more extreme) personal experiences and subsequent explanation, and then the opportunity to see so much further into the city, as an icon of Indian life. This is classic participant-observation. What’s the Hindi word for gonzo? How’s about gonzoti?

There’s a lot of exuberance about India nowadays and I think that needs to be tempered with some other perspectives. It’s not necessarily an easy place to live, work, visit, or develop.

From our Design Research Methods class, some observations from an interviewing exercise.

The scenario was to conduct interviews in order to uncover opportunities in helping people to manage food, meals, nutrition, etc.

The question asked was
What are the challenges you face in meal preparation?

Of course, that question is flawed because it presumes that there are indeed challenges. This was evident when the respondent struggled with how to answer outside the frame of the question.

An alternative might be
Are there any challenges in meal preparation?
which is is more open-ended.

But better still is
What are your feelings about the experience of meal-preparation?

since it doesn’t put the label challenges into play. It would be important to understand the labels the person being interviewed places on the different aspects of their experience, and to use their terminology to probe further.

Also worth noting is that the original question came right off the sample interview guide I distributed. Sometimes the interview guide is a tool to document “questions you want answers to” rather than “questions you want to ask”; doing fieldwork involves a lot of translation back and forth between the two.

Yesterday on KQED’s Forum their guest was Brooke Gladstone from On the Media. I was struck by this quote, early in the broadcast.

I will ask a question…maybe 5, 6 times, and the person we’re interviewing gets around to answering bits and pieces of it after every iteration and we will sometimes assemble all those answers into one.

This is the art of the interviewer — probing, following up, asking again, in order to get to the answer, eventually. It’s why interviewing in teams is tricky, because the other member of the team needs to know what’s in your head, to give you space to ask that question over and over again til you get to it. Regular people do not speak in soundbites.

We also see this over and over again when we’ve synthesized conclusions and we go look for supporting video data to share with our clients. It’s just not there as some obvious artifact. That’s why, of course, our recommendations come from synthesis — insights are not sitting there waiting to simply be scooped up, they have to be assembled from a variety of sources.

I am envious of those with amazing audio/video editing chops who can use the data to more closely approximate what we want to say; but at the end of the day, the big ideas rarely come out of people’s mouths directly.

One week from tonight is the first class in the Design Research Methods that course I’ll be teaching at the Involution Master Academy in Sunnyvale, CA. If you’re thinking of signing up, don’t wait!

I hope to see some of you there.

I just started reading Guerrilla Marketing and was bemused by the list of associated books.

It’s way too small to read here, but click the picture to see it larger.

Man, am I late to the party. I only just read “Blink” but it was more than 2 years ago when PeterMe wrote

Everything You Need to Know About “Blink” Boiled Down Into 9 Words.

Snap judgments are valuable. Except when they are not.

I’d add 6 qualifying words: Hindsight is 20-20 in determining which.

We’re doing a bunch of fieldwork these days looking at how people are using their software and hardware to accomplish some tasks. It’s interesting to see the difference in the flow between the part where we ask people to describe what they do and how they do it, and the part where we ask them to show us how they do it.

The discussion part is hard. It’s a bit abstract to explain a detailed behavior in absence of any props or artifacts. People work to give clear explanations and it takes a lot of follow-up to get the details.

Most of the same people light up when they are asked to show us (although some simply decide to show us without prompting).

But I find myself liking the “tell me how” part of the interviews better; the comfort level is lower, but the struggle to articulate is very insightful. Looking at how people describe things from memory isn’t wholly accurate in capturing their perceptions or usage, but it pulls out some neat contrasts.

And ideally, we’re trying to get stories, not factoids. The discussion (not the demo) is much better for stories. The participant takes over in the demo, and it becomes a semi-hurried list of “this works like this, and this other thing works like that.”

Of course, we’re doing both, and we need to be doing both, and some of the insights will come of the tension or differences between the two.

I think we’ll switch the order on today’s interviews, and maybe try starting with the demo, and then doing a discussion afterwards. I sometimes feel the demo requires time, and rapport, and trust, before we can safely ask for it (especially if the equipment in question rests within an inner sanctum), and so this is a bit of a leap for me.

Of course, there are few “right” answers in evolving one’s technique, it’s about building up a larger palette of approaches and making intelligent choices about when to switch around. I’m not at all unhappy about how the interviews have gone so far; they’ve been fun and fascinating, but I’m thinking hard about how to keep doing better.

Recently we were interviewed for two different global studies on global innovation. Both the German Research Foundation or Deutsche Forschungsgemeinschaft and the Danish Ministry of Economic and Business Affairs and Centre for Economic and Business Research or FORA separately came through the Bay Area as part of as study of leading innovation/design entities in the US and around the world.

Both groups offered to share their results with me and so I’ll post more specifics here when I’ve got ’em. Meanwhile, it was fun to be interviewed instead of doing the interviewing, even if it was clearly a conversational, expert-interview type of thing, and not an ethnography (neither did a huge amount of following-up or probing, it was pretty much based on what I had to say). One interesting technique, though: the group from FORA asked me to reiterate some of my key points from our longer interview (recorded on audio) for a 5-minute summary video. They went over with each other, and with me, what points seemed the most salient, and then reinterviewed me into the video camera. The point was not to get more information, but simply to create an effective deliverable. I was not at all uncomfortable with it, indeed I felt very listened to in having a precis of my interview created on-the-fly. It reminded me that being able to communicate what is learned from the field is crucial; that methods could be innovated around the effective communication as much as the effective discovery.

This comment in the bad survey design thread got me thinking further about where/when/what to do with surveys. It’s not my primary tool so some of these reflections take me a little longer than someone who makes their living as a quantitative researcher, for example.

A tiny new restaurant opened in our tiny town of Montara – the Montara Bistro. I dropped by yesterday to pick up a menu and saw that they are already looking for customer feedback.

So to some folks, this is a survey. But it’s next to useless.

Why? Their questions are not too bad, but they are conversational questions, and should be presented that way. They are the basis of a conversation. Handing someone a sheet of paper (with no room to fill in any response) and asking them for essays is ineffective. It’s not fair. These are the questions they want answers to, but sometimes you have to ask a series of questions to get that information. And you can’t decide ahead of time which questions to ask. You have to ask a question, listen to the response, and then choose your next one. You can’t do that on a piece of paper. You need to have real people talking to each other and exploring the issues that way.

Not to mention that the restaurant has been open for a day or two, and there’s a presumption of an in-depth relationship that hasn’t really been built yet. What do I think of the Bistro Vision? Ummm, I don’t care.

I love what this artifact tells you about the company; that they really want to get a dialog going. They don’t have the tools in place to do it yet. Maybe it’s backed up by the way they interact with customers who come in; I don’t know. But this won’t work at all.

And I think this sort of inquiry is what a lot of design students are doing; identifying some open-ended (i.e., requires the respondent to write sentences) questions and sending them out by email. Some people will respond. Some may even write a lot. But you can’t follow up unless you send out another email. And then it’s just a conversation.

As with everything you “send out” who it gets sent to is a factor. Sending something to 3 friends is a very different approach than something that is quantitative in nature.

Look at this artifact from a recent project (created by our partners, not us):

This contained 31 questions, only a few open-ended ones. There’s randomization where needed (so you can filter out order-effects, where the first or last item might be picked more frequently in a list), and a large enough sample so that results can be processed to lead to conclusions – comparisons between different factors (this is the stats part I’ve been talking about).

Attitude toward technology meets Age
Purchase habits meets Region (with Age)
Stores shopped meets Region (with Age)
etc.

Tons of work and tons of math goes into creating tables (that then get interpreted) like

As Paul Hogan (sorta) said “That’s not a survey, now that’s a survey!”

I hope this brings a bit more clarity to the discussion.

Judging by the cover of The Houdini Solution, inside-the box-thinking is the new out-of-the-box thinking. Whaaaaa? That’s so counter-counter-intuitive that it must be genius.

Right.