Weblog Consulting Register Login Subscribe to Free Influx Emails Join our Facebook Group Branding Street Cred Technology Lifestyle Style Entertainment Interviews Art InfluxTV Map Widgets RSS — All Subjects RSS — More Feeds

Search

Advanced Search

Contact Privacy Policy

People have been using the web to do consumer research for over a decade. Its had its fair share of problems especially in quant, where it's already been proven that only a very distinct group of consumers ever complete surveys online.

However, as people become comfortable with sharing more and more of their lives online, it's becoming a very interesting place to get a feel for what people are thinking and doing. People are using blogging tools and media uploading tools to help them get a better understanding of their consumers and these are often richer and more insightful than the typical focus group.

With 2.0 technologies, all this only going to improve.

One example I really like is Noah Brier's Brand Tagging project. This simple application that allows people to enter words they associate with specific brands

and see the results from others.

What I like here is that the tool employs some good thinking. It feels more like play, than research. It doesn't take a lot of time and who doesn't want to be rewarded after they've contributed by seeing everyone's results.

Obviously, Brand Tagging isn't going to replace anything fast, but it illustrates the power of a relatively simple and low cost tool to generate some useful insight.

It would be great to hear of more examples of this type of thing


Posted by Ed Cotton Arnold creatives, Roger Baldacci and Lawson Clarke made this film for the Hatch Show, it's a rather alarming take down of focus group testing .



I traded a few emails with Roger to learn more about how the film was made and what inspired it.

1. Tell us your background- years in the biz etc?

I’m an Executive Vice President, Creative Director at Arnold. I currently run the ESPN business at Arnold and also work as a senior writer, partnered with art director Paul Renner. I’ve been in the business about 17 years.  Prior to Arnold I worked at Fallon in Minneapolis.
 
2. What inspired this labor of hate, if you are at liberty to tell us?

I call this a labor of hate, because I simply hate concept testing with focus groups.  I have witnessed untold creative concepts die at the hands of people who are only there for the $50 and free M&M’s and Pepsi.
 
There are just so many flaws to the system that I believe it’s not scientifically valid on any level.  These flaws include:

1. Moderators can lead respondents
2. Alpha respondents can lead/sway the rest of the group
3. Spots that are easiest to understand/most familiar often win out
4. Concepts aren’t accurately reflected with cartoon storyboards
5. The mindset of respondents is to criticize no matter how objective you ask them to be. It’s human nature,
6. It's not reflective of the media buy. I’m in this business and I can’t tell you what commercials I saw last night. Brands are built by a cumulative effect on multiple sources.  You can’t expect brand preference to go up after ONE viewing of a bad animatic.
7. Time consuming. Creating these takes time and energy from the creative department.
8.Cost prohibitive. It’s not unusual to spend $300K to find out
9. Preconceived notions. Clients and agencies tend to have ideas of what they want to get from the focus groups so they look for those. One client felt we had a brand linkage problem when the moderator asked the groups to name a commercial in the client’s category. They didn’t name our commercials. I said that if they recalled our commercial, but couldn’t remember it was from us—that’s a brand linkage issue. But they couldn’t even name our spot which means we had a media issue.
10. Kills the creative process. If a spot tests well in groups, clients are unwilling to deviate from it when it comes time to shoot. So trying to get a good director to plus the boards? Forget it. The magic of ad-libs from the talent on shoot day? Forget it.
 
But let me be clear, I’m not against focus groups and learning information about your target audience. I think if you poll planners in all agencies, they would much prefer to do up front market research over concept testing. The issue I have is with testing unfinished work in front of people.
 
Sure, the stakes are high for CMO’s with the average tenure being 22 months on the job before they hope to take their success and move on to another job. But look at the movie industry. The average development costs for a typical Hollywood movie is around 100 million. Do the studios do animatics or board o’ matics of their movies with tests audiences to see which movie they should produce and how best they can “optimize it?” Of course not. They green light a script, film it, edit it, mix it and THEN they have test screenings. Now they’re just asking to see if there was something the audience didn’t get or found confusing. This may result in re-editing it or perhaps reshooting one or two scenes.

But imagine showing an animatic of the film Adaptation or Eternal Sunshine of the Spotless mind to test audiences?
 
3. Why did you pick the Apple spot? Were there any other candidates?

We thought about other high profile spots that people within the industry would know, but in the end, we decided to go with the most iconic spot in the history of advertising. There was something so poetic about taking such a high profile Super Bowl spot for a visionary brand like Apple, (who don’t test, clearly) and subject it to the whims of people who need extra cash to pay their cable bill that month.
 
4. How did you make the film?- do the recruit and my guess is that's not an Arnold planner in the film....

We used a real focus group facility who donated the time and recruited real respondents at cost. The moderator was an Arnold planner who recently took a job on the West coast. We briefed the respondents exactly the way we would a normal focus group—telling them that we “had nothing to do with this commercial” and that we were looking for their “honest feedback, both good and bad.” We told them this was a potential Super Bowl spot. We had two cameramen in the back room behind the mirror, each responsible for close ups on one side of the table. For the majority of the shots, however,  we used the actual focus group camera and audio from the facility.

5. Do you think creatives believe all focus group testing is like this?

The film has been posted on many blogs (in many languages) and you can tell by the responses that this has struck a nerve with many creatives. We have all been there and we’ve all see this. So, yeah, this is pretty much how all (concept testing) focus groups tend to go.
 
6. What does it feel like when you have to sit in the other room in a session like this and listen to people talk about your work?

Any good creative has a thick skin, so hearing your concept get ripped to shreds is not new.  You smirk. You make witty, sarcastic remarks about the respondents, down a fistful of salted peanuts and move on. 
 
Frankly, if focus groups liked your concept then maybe you’re doing something wrong.

Again, focus groups typically reward the familiar. I have run into this situation many times. One time, the agency recommended spot lost out to the back up spot because it featured a husband speaking to his wife in the kitchen. Very easy to understand. The recommended spot featured intricate cg work and was actually more on strategy than the husband/wife spot. We produced the spot that “won out” in testing and the client proceeded to do everything they could in the edit to make it more like the spot that died under the harsh florescent lights of the focus group room.
 
7. How have people in Arnold reacted to the film?

People have loved it. Both Lawson and I have received positive feedback from inside and outside the agency walls. It has been forwarded to many people in the business, including the production side. People dig it because it’s so painfully real.   
 
8. If you were a planner, what would you do to help creative teams develop great work?

The biggest mistake planners make is to pretend they’re in the creative department. If you want in, by all means put your book together and jump in. There’s plenty of room behind the two way mirror at the testing facility.
 
What creatives are looking for from planners is a simple, unique insight into the human condition of our target audience. Granted, that’s not always easy. But telling creatives that teens like music and technology is not helpful. Don’t give us charts and graphs because we know we always want to be in the upper right hand quadrant.
Give us a simple sentence that can get our gears turning. Tell us something we didn’t know about our audience. We all know that milk is good for your bones and does a body good. Just hand us a postcard with a sentence like “You know. It really sucks when you want milk and you don’t have it.”
 
9. What stuff is inspiring you these days?
 
There has been very little that has inspired me within the advertising industry lately. For an industry that prides itself on originality, it’s amazing how often we copy each other.  Remember shaky cam? Remember large type to small type? Remember the “organic” layout with a bunch of stuff photographed on the page with copy wrapped around it? Remember binding in little booklets into magazines? Remember that whacky, offbeat Swedish humour?
 
I tend find inspiration from films, photography and contemporary art. I know that sounds pretentious. So if I had to pick one commercial thing that inspires me I would say the writing of screenwriter Charlie Kaufman and also the writing staff of NBC’s “The Office.”

Thanks to Katie for finding the film. We are looking forward to the sequel on quant pre-testing!
 


Posted by Ed Cotton ...it actually wasn't meant rhetorically in a recent survey by the Pew Center in Washington.

The conclusion was that there is a 14% increase in the proportion of Americans who say environmental problems are indeed a major global problem (23% in 2002 to 37% in 2007).

U.S. concern is low when looking at other advanced industrial countries, especially since we're seen by 34 of 37 countries surveyed as “hurting the world’s environment the most,” but the level of increase is still comparable.

Since the environment has been such a hot topic recently, perhaps the 2002 numbers tell more of a story. For example, even though China’s concern only rose 1% since 2002, 69% of Chinese people were concerned about the environment that year. This was the highest proportion of concern of any country in 2002 except for South Korea (73%), and Chinese citizens were three times more concerned than Americans that year.

There is evidence that China's leaders are catching up to the concerns of its people, even if only as a result of outside forces, but environmental initiatives are continuously canceled out by colossal economic plans.

For instance, new vehicle-emissions regulations are in the works, but they cannot match the exceptional rate of growth in the number of cars (projected to surpass the U.S. at 130 million vehicles by 2020), due to a policy to promote the domestic car industry.

So when will the U.S. approach the higher echelons of concern, if ever?
Hopefully our 14% increase signifies something bigger: that the level of concern in a country won’t always have to correspond with its level of pollution.
 

Posted by katie facada "We built the store inside a warehouse. We claimed it was a movie set so that people would deliver all these goods and not think it was us. We had to borrow products from other retailers because we hadn't at that stage developed any products. We took ordinary people in, and they really, really liked it."

Sir Tim Leahy-Tesco (on his company's research for the US launch)


Posted by Ed Cotton This month's Fast Company has an article about the resurgence of Disney. However, reading a recent blog post from FC about the President of ABC-Disney television group, Anne Sweeney, Influx wonders were the insight comes from.

The post references Anne's recent trips into the field and onto college campuses, where she witnessed first hand the changing media habits of this new college generation.

"Even on days when she's being a mom instead of a high-powered executive. Sweeney, who's based in Los Angeles, was on the East Coast primarily to visit college campuses with her daughter. Seven colleges in a week. Sweeney couldn't help but notice how ubiquitous laptops were and which shows resonated with students (the lounge TV on one campus had a sign that read, "Reserved for Lost"). "The trip has definitely given me a view of this younger group's experience with media that I don't normally get," she says. "It's first-hand market research. I've got a thousand ideas from this."

Not to mention a new appreciation for that generation's WiFi expectations. "Last night, we checked into the hotel, my daughter said, 'Oh, I’m so happy.' I said, Because you’re in New York? She said, 'No, because I finally have a signal.'"

Sweeney had to grin. "Yeah, that’s what I said to my mother on my college tour."

It leaves us thinking:

1. Should we assume that all this was new news to her?

2. Based on this illuminating experience, shouldn't she be getting out of the office more often?

3. Shouldn't her research folks be doing a better job of bringing their targets to life?

This is a tale about experiences counting for far more than listening to a research debrief.

It's a big challenge for all planners, agencies and consultants.

How do you get the client to understand it, if they don't experience it?

You have to try and make the client a part of the process and you have to bring creativity to your findings; bring them to life for those who didn't experience it first hand.

Make the research, creative.


You conduct segmentation studies, focus groups, surveys to try to learn who your customers are. You want to create segments, groupings, tidy little boxes to put them in, but customers are more complicated than you want them to be.

Here is Dave Pollard's nice illustration showing what defines us.



While the closest you might come to undestanding your customer is to have 12 of them in a room for 2 hours. How much can you get to know? Your brand might be simple, but they are complex and their complexity determines the relationship they have with you.

You don't get this from web surveys or profiling.

You don't get the nuances from data.

It's time to get out and spend some quality time with your customers. Go on a trip together, visit their homes, go shopping, drink together, cook together, live together. There are endless possibilities, if you are prepared to make the effort.