Gerry Gaffney runs the UX consultancy Information & Design in Melbourne. He publishes the User Experience podcast and is current director of publications at UXPA.
I was researching, with my colleague Patrizia Bordignon, how people thought about and dealt with home renovations.
One of the methods was a diary study (“cultural probe”), and we had carefully recruited – or so we believed – a small set of participants with whom we would work for several weeks.
Warning bells sounded fairly early with one of the participants, who showed up very late for the initial briefing. These things can happen, so we ran a separate briefing session for him, gave him his kit of reporting materials (camera, diary and so on) and sent him on his way. Let’s call him Mr. W.
Three days after the briefing we telephoned each of the participants. It’s a good idea to do this to remind people about their commitment, to redirect as necessary, and to address any issues that arise. All our participants were on-track, with the notable exception of Mr. W, who seemed somewhat evasive in his answers.
At the end of the first week, we visited the participants. Again, this is good practice; it’s an opportunity to see how the data is being gathered, and what changes might be needed to the process. We also use that opportunity to make a part-payment to the participants, which can serve as a nice motivation.
We were delighted with what we saw. Participants had kept bills and receipts, photographs and magazine clippings, they showed us their renovations or their plans, and we were confident that we were getting plenty of highly relevant data.
When we visited Mr. W’s house, however, it was evident from the first moment that his home was different. The front gate didn’t work properly and the hinges squeaked, the garden was unkempt and the house gave an overall sense of dilapidation. Inside it was a similar story. Every room was in dire need of immediate restorative work, but none was evident. I felt a tad depressed as we drank tea from cracked mugs and listened to Mr. W list the things that needed to be fixed.
Mr. W was not an enthusiastic renovator. His house represented a series of urgent and necessary tasks, none of which had been tackled.
It looked like we would collect no useful data from Mr. W, and as we traveled back to the office we talked about our disappointment and reexamined our recruiting strategy.
However, as we moved into data analysis, we found ourselves referring quite often to Mr. W, and gradually came to realize (no doubt this should have been obvious earlier) that Mr. W’s world was in fact directly relevant to our project. While the enthusiastic renovator was undoubtedly a key consideration, the unenthused or reluctant could also present great opportunities. Their needs and goals were different, their attitudes were different, and the way that we would design for those characteristics was different.
In many ways, in fact, Mr. W was an ideal participant specifically because he didn’t fit our expectations. He challenged the underpinnings of the project, and made us examine our design decisions in a much more rigorous fashion.
I often reflect back on this experience when I’m doing user research, and I specifically watch out for negative reactions and experiences, because they can often teach us things that we might not otherwise learn.
I still believe it’s important to recruit carefully, but perhaps we should be more open to the idea that the “wrong” participant is sometimes precisely the right one.
