Posts tagged “testing”

ChittahChattah Quickies

  • DEVO – Focus Group Testing the Future [YouTube] – Filled with brilliantly sarcastic soundbites, this is definitely pushing on post-modernism/post-irony. DEVO doing focus group testing (or so they say) on every aspect of their 2010 offering (brand, logotype, instrumentation, clothing). Interesting also to see how this appears in the press with varying amounts of the irony removed.
  • Theater Preshow Announcements Take Aim at Cellphones [NYTimes.com] – In a production of “Our Town” the director, David Cromer, who played the Stage Manager, took a minimal approach because he wanted to stay true to Thornton Wilder’s desire to forgo conventional theatrics. “In that show we had this issue, which is that there was to be no theater technology. The whole act of my entrance was that you were supposed to think it was someone from the theater,” Mr. Cromer explained. “We didn’t want the Stage Manager to come out and say, ‘Please turn your cellphones off,’ because that would be rewriting Wilder.” Instead Mr. Cromer simply held up a cellphone upon entering at the beginning of each act and then turned it off and put it away, casually showing the audience what to do without talking about it. “The first time I was watching another actor take over in the show as the Stage Manager,” Mr. Cromer said, “he came out, held his cellphone in the air, and the woman next to me said, ‘Oh, someone lost their cellphone.’ ”

ChittahChattah Quickies

  • Listening to customer feedback? Twenty-Five Years of Post-it Notes (Thx, @susandra) – In '77, 3M decided to test-market. It failed to ignite interest. “When we did the follow-up research, there just weren’t a lot of people saying this was a product they wanted.”
    "We knew the test markets failed, but we just kept saying, ‘Maybe it was us. Maybe we did something wrong. Because it couldn’t be the product—the product was great.”
    To see for themselves how people responded to Post-it Notes, 2 execs cold-called offices, giving away samples and showing people how to use 'em. The responses were more enthusiastic. “Those things really were like cocaine. You got them into somebody’s hands, and they couldn’t help but play around with them.”
    1 more test was in order. They got newspapers to run stories about it. They festooned stationery stores with banner displays and point-of-purchase materials. 1000s of samples were sent to office managers, purchasing agents, lawyers, etc. People demonstrated it to potential customers. It was a huge success, and 3M decided to launch Post-Its.
  • Listening to customer feedback? Peter Arnell Explains Failed Tropicana Package Design – Big outcry over the Tropicana packaging design (which this suggests was NOT tested but that's hard to believe) led to a return to the previous packaging.
  • Listening to customer feedback? Malcolm Gladwell on the Aeron chair – The Aeron chair was originally despised and deemed ugly. It didn’t catch on for 2 years, and then it quickly became the most popular chair. Everyone came to love it. Gladwell concludes that people find responses about some topics extremely difficult to articulate. While they may think they dislike something (like the Aeron chair), in their hearts they may actually like it. There is a disconnect that causes people to express dislike in their heads while they actually like it in their hearts (and vice versa).
  • Listening to customer feedback? Hate Facebook's new look? You'll like it soon enough. – Slate advances the point that people react to change negatively but eventually get used to the change and make it work.
  • Listening to customer feedback? Problems With NBC’s ‘Parks & Recreation’ – When do you listen to negative feedback and when do you follow your vision? I think there's an important middle-ground that is often ignored: understanding what lies beneath that feedback and choosing carefully if and how to respond to it, or how to create supporting activities that help get over the barriers that the rejection points to

ChittahChattah Quickies

  • Report: Real-world police forensics don't resemble 'CSI' – Even before the popularity of shows like CSI, there was presumably a cultural belief in the "science" behind these techniques. But the report finds that:
    – Fingerprint science "does not guarantee that two analysts following it will obtain the same results."
    – Shoeprint and tire-print matching methods lack statistical backing, making it "impossible to assess."
    – Hair analyses show "no scientific support for the use of hair comparisons for individualization in the absence of (DNA)."
    – Bullet match reviews show "scientific knowledge base for tool mark and firearms analysis is fairly limited."
    – Bite-mark matches display "no scientific studies to support (their) assessment, and no large population studies have been conducted."
  • NJOY electronic cigarette – Looks like a real cigarette, complete with glowing tip on inhale, and exhaled vapor that resembles smoke. Gives an inhaled nicotine experience, while messaging to the rest of the world that you are really smoking a real lit cigarette. Paging Erving Goffman?

    Someone was using one a party last week; someone else got out their simulated Zippo lighter (an iPhone app) and lit it for them.

Trying to find out things we didn’t even know to ask about

Great article about Disney theme parks and the design – prototype – test – iterate – build process for a new ride, Toy Story Mania

“It is much easier and less expensive to do this before the concrete has been poured,” he added. “As rides become more complicated, your ability to tweak in the field gets harder and much more expensive.”

Across the street, in a cold, unmarked garage, Ms. Allen helped to conduct “play tests” on rudimentary versions of the ride. More than 400 people of all ages – all had signed strict nondisclosure agreements – sat on a plywood vehicle set up in front of a projection screen and played various versions of the games. Disney workers studied their reactions and interviewed them afterward.

“We were looking to see if some effects were too scary,” Ms. Allen said, “or if there wasn’t enough laughing happening during certain sequences.”

Among the discoveries: People wanted to be able to compare scores after they were finished playing, while some children had a hard time reaching the cannonlike firing controller, christened by Disney as a “spring action shooter.” Engineers added a computer screen to vehicles to display scores and installed the controls on movable lap bars.

“We were trying to find out things we didn’t even know to ask about,” said Sue Bryan, a senior show producer.

Passing the buckskin

Lululemon got into trouble last month for selling clothing made from a seaweed-based fabric that supposedly had many unique properties. But independent tests revealed that the material was just cotton, and performed like cotton.

Mr. Wilson added that the company probably did not have enough money to test the material back when it started using it 18 months ago. When asked about Lululemon’s product tags and the claims about vitamins and minerals, he said, “That’s coming from the manufacturer. If you feel the fabric, it feels a lot different.”

And it gets even more laughable in the followup story

When told earlier this week about the Times’s test, Dennis Wilson, the founder, chairman and chief product designer, said: “If you actually put it on and wear it, it is different from cotton. That’s my only test of it.”

Research screening

I was bemused to see that Feast of Love opened last weekend. Our last time at the movies was when The Simpsons Movie opened, and I participated in some intercept-market research at the theater.

Part of the lobby had been given over to these groovy looking kiosks, with a couple of guys in attendance, asking people who passed by if they would like to give their opinion about an upcoming movie. My age and gender qualified me to participate (woo hoo) and I went with one dude over to a kiosk. I was shown a couple of clips and responded to various questions, but the weirdness of it was that the test was designed have some screens operated by me, and some screens visible only to the interviewer. But they didn’t do it that way. So for various pieces where I was to click within multiple choices, the interviewer, who knew the testing software rather well, just whipped through the keypresses, bam->redraw, bam->redraw, quickly asking me the minimum to move to the next one. Okay, so he took care of it for me. But then this screen we were both looking at would display testing instructions such as ASK PARTICIPANT FOR OPINION OF BENEFIT OF DATE MOVIES. PROBE ON RELATIONSHIP, TIMING, COST. And of course, he wouldn’t even come close, he’d get the one line answer from me, and then he’d type in the quickest condensation of my answer: stay home.

After a minute or so, it became more about the two of us cooperating to use the software to get through test. I realized that my opinion didn’t matter; it’s hard to feel represented in a forced-choice discussion, and it’s unlikely one would continue to provide color when all that gets captured is minimal facts. Further, by exposing the instructions to me, his shortcuts became clear, and I ended up slightly co-opted into the testing process, giving up any sense of really delivering the full truth to this interviewer.

When we see “market research” number published to support some business decision, let’s keep in mind how poorly that data may have been collected (from the concept of how to collect that data, to the implementation of a data collection environment, to the staffing and execution of the data gathering). How reliable could any of this possibly be?

Do you recall?

Some recent product recalls

Wild Planet Toys Inc. of San Francisco is recalling 273,000 Jet Streamers Water Blasters pool toys. When partially filled with water, the pool toy can stand upright on the pool floor with the rigid narrow end pointed upward, posing an impalement risk.

— LeapFrog Enterprises Inc. of Emeryville is recalling 186,000 Playground Activity Centers. A child’s arm can become caught in the activity center’s plastic tube.

— Olympus Imaging America Inc. of Center Valley, Pa., is recalling 1.2 million Olympus 35mm film cameras. A defect with the flash circuit in the cameras can cause it to smoke and overheat when the camera is turned on, posing a burn hazard.

— Syratech Corp. of East Boston, Mass., is recalling 10,000 frog, fish and duck lawn sprinklers. The plastic body of the lawn sprinkler can crack when placed under intense water pressure, and pieces of it can break off and be projected 5 to 10 feet in the air.

— Pier 1 Imports (U.S.) Inc. of Fort Worth, Texas, is recalling 4,300 Ming television stands. If a person leans on the stand’s drawer when open, the unit can tilt forward and cause a television on top to slide off, posing a risk of injury or death.

— Ballard Designs Inc. of Atlanta is recalling 775 candles and candle sets. The packaging or holder can ignite, posing a fire hazard.

— Agio International Co. Ltd. of Hong Kong is recalling 33,800 Garden Treasures steel dome fireplaces. Touch-up paint used on the fireplace’s exterior can ignite during use, posing a fire hazard.

— True Religion of Los Angeles is recalling 150 hooded fleece jackets. A drawstring is threaded through the hood, posing a strangulation hazard to children.

— Onward Manufacturing of Waterloo, Ontario, Mi-T-M Corp. of Peosta, Iowa, and Deere & Co. of Moline, Ill. are recalling 3,100 John Deere gas barbecue grills. Operating the grill in windy conditions can blow the flame under the control panel, causing the grill to overheat or cause flashbacks. Flames could damage the hose that supplies gas to the burner, causing an uncontrolled flame. Also, the grill’s control knobs could overheat, resulting in burns to hands.

— Deere & Co. of Moline, Ill., is recalling 16,000 John Deere X300 Select Series lawn tractors. A problem in the manufacturing process could cause damage to the circuit in the interlock module. If the module fails, the mower blades will be able to run with no operator on the tractor seat.

— Kindermusik International Inc. of Greensboro, N.C., is recalling 10,000 cage bells. If the bell inside the instrument is damaged during manufacturing, the bell can be pulled out of the instrument, posing a choking hazard.

— Triangle Tube/Phase III of Blackwood, N.J., is recalling 3,000 water heaters. The burner plate and flue hood seal on the water heaters can fail due to an improper seal, causing a leak of flue gases and deadly carbon monoxide.

— Gotham Architectural Lighting, a division of Acuity Lighting Group Inc. of Conyers, Ga., is recalling 4,700 lighting fixtures. The reflector/trim pieces may not be properly attached to each other. The lower portion of the reflector/trim assembly could detach and fall from the ceiling, striking people below.

Of course, being injured by a product you’ve purchased is not funny, but something about the tone of the descriptions is funny (if you find police blotter sections of local papers funny, then you’ll know what I mean here) in a Simpsonsesque fashion (or the famous German Forklift Safety Video).

And in blogosphere synchronicity, Niti’s last story in this post is a slightly more sober take on a product recall.

A test post

The new blog is here!

Not only a new blog, but now the entire site is in WordPress – this makes for some interesting things (like trying to have separate feeds for FreshMeat and this blog; or titling this page “All This ChittahChattah” while titling the rest of the site “Portigal Consulting”).

FreshMeat is definitely not working right now, but maybe within 24 hours we’ll have them all back up and sorted out (they are still up, under the old URLs, but whatever)

If you see other weirdness with the site, let me know

Series

About Steve