Pre-testing ads doesn’t kill creativity. Apparently.

April 20, 2010 1 Comment

There’s a guest post on Mumbrella titled ‘Pre-testing ads doesn’t kill creativity’, from Darren Poole, Millward Brown’s Chief Client Officer.

The post was written in response to comments from Geoff Ross that “… advertising is completely in a quagmire, multiple layers, pre-testing, post-testing. In the end, nothing good is going to survive”.

Darren sets out to articulate why this isn’t the case, and why pre-testing delivers significant benefit to the creative process.

I want to find something illuminating in what he’s saying. I’m tired of the utter predictability of the ‘pre-testing is nonsense’ response. I worry that maybe disparaging research is my obligation as a member of the advertising industry (like over-using Hipstamatic or coveting Moooi).

But try as I might I can’t find anything here that changes my view. In fact he makes a couple of comments that only reinforce it. One is this:

‘While none of us like the idea of someone evaluating our work (hands up who loves performance reviews?) the reality is that advertising is a most public profession and the target audience is the ultimate judge of creativity.’

I think this rather succinctly captures the central issue.  Because the target audience isn’t the ultimate judge of creativity.  The target audience is the ultimate judge of persuasiveness.  It’s the failure to grasp the distinction between these two things that is at the heart of the pre-testing problem.

Clients don’t make ads to be creative.  They make them to persuade. Creativity is just the means to the persuasive end.

And the reason this is important is that people don’t really like the idea that they’re being persuaded. They actively avoid the sense of being manipulated – so much so that they will deny it ever happens.  So an effective ad has to persuade covertly, meaning that a great ad will be persuasive for reasons that people struggle to identify and explain.  It’s the unexpectedness, the under-the-radar stealth that makes a great ad persuasive. So a great ad has to be persuasive before anyone realises they’re being persuaded.

Which is why you can’t make the target audience the ultimate judge of creativity.  Not because, as is often suggested, they know nothing about advertising or creative. But because they know something about advertising and creativity.  So when you ask them what they’d do to an execution they’ll happily tell you. They’ll give you lots of helpful suggestions based on things they’ve seen and liked before. But this can only mean (and I think you can see where I’m going with this) that they’re all things they’ve seen before, making them less persuasive, and therefore less effective.  So if you identify a problem with the persuasiveness of an execution, the creative answer is not to try and persuade in a more obvious way. The creative answer is to try and persuade in a more original way. And that originality is not going to come from the target market.

I have a second issue with Darren’s argument. He gives a series of examples that highlight the improvements made to some executions that Millward Brown has pre-tested.

“I can’t give away too much detail, but some of the work we have done with CPG brands in the past 12 months has seen a soundtrack transform from something viewed as a little strange to become quirky and impactful. Another great example is where we helped characters evolve from being creepy and a bit disturbing to wonderfully eccentric. We also indentified the route to help a brand develop one of the most popular ads of the Summer, and we’ve helped make a brand’s consumer benefits more evident in more than one case’.

I don’t doubt that there are occasions on which pre-testing has molded a soundtrack or character into something that has subsequently worked.  Darren proudly, if vaguely, lists some examples of how pre-testing has improved the quality of some advertising. But I also don’t doubt that there are a great many more occasions on which pre-testing has squeezed the life out of something that was potentially interesting – times when it’s taken what could have been a quirky and impactful soundtrack and turned it into something familiar and innocuous, or turned a potentially eccentric and memorable character into a boring and forgettable one.  Pre-testing has consistently resulted in advertising that is more predictable and less original, in the process making it less persuasive and enjoyable. And on that basis I suppose you could argue that pre-testing has improved advertising, in the same way that you could argue that Simon Cowell has improved music.

I think the advertising industry embraces the idea that research can help us understand how the target audience might be persuaded. We’re very happy to learn more of how the audience feel about the issue at hand. (We’ll even put aside the fact that they, like you and I, are transparently incapable of describing their genuine feelings on most subjects, quite unable to rationally explain our irrational attitudes.) But if research can provide some clarity here, let’s have it.

But what we don’t embrace is what Darren’s note exposes.

The research industry’s belief that the target audience should be the judge of creativity is flawed.  The judge of persuasiveness, perhaps, but please don’t confuse the two things. And the research industry’s belief that the pre-testing process consistently highlights ways of improving ads is equally flawed.  We know pre-testing consistently highlights ways of changing them. But improving them? That’s another issue entirely.

(Two interesting, and contradictory, pieces here from WARC. The first details a TNS study showing the improvement in effectiveness delivered by pre-testing. The second shows exactly the opposite.)

Filed under Research Tagged with , , , , ,

Focus groups. Your neighbour’s opinion. And witches.

December 22, 2009 Leave a comment

I was sent this article from Advertising Age the other day. It revisits an issue everyone in our industry is painfully aware of – the puzzling tendency of clients to rely on focus groups to inform marketing decisions.

I sent the article to a friend (who happens to be a senior marketer) to see what she thought.   Her response was interesting.

I asked her whether it might be a little artificial to ask people to explain the rational motivation for a purchase decisions when those decisions are made without any great rational thought. She said that I was trying to make the process more complicated than it actually is and that most people are quite good at explaining why they make decisions.

I questioned her as to whether asking people with no experience whatsoever in design or communication to comment intelligently on important issues of design or communication might not be a bit optimistic.  She countered that ‘agency people’ only want to design stuff that other ‘agency people’ like and that talking to real people is a necessary reality check.

But the final comment she made was the most interesting.

She said ‘Marketers don’t buy shampoo.  My neighbour does.  So I thinks it’s useful to get my neighbour’s opinion’.

Which on the surface seems like a reasonable view to take. It’s ‘ordinary’ people who buy stuff, so the view of ‘ordinary’ people should be relevant.

But the argument falls over for a few reasons (the first two alluded to above).

Firstly, most purchase decisions are emotional. Research seeks to explain them rationally.  Hasn’t neurological science established that the two processes are entirely different?

Secondly, people who buy soap aren’t qualified to design packaging or advertising for soap, in the same way that people who attend rugby matches aren’t qualified to coach players or design stadia.

But I think there’s a bigger issue that I’ve always struggled to articulate.  I can’t understand why we put people we know nothing about in a room and unquestioningly give credence to their views on a very important subject (to us at least).  These are often people who, if we met them outside the confines of a meeting room, we would possibly think were less than, shall we say, credible.  The fact is we know nothing about their view of the world, but we put them in a research group and assume the capacity for sagacity.

Then I read this at the weekend.  It’s a summary of what, according to recent research, Australians believe in.

Amongst other things, 51% of Australians believe in angels – winged messengers of God sent to earth to perform specific tasks at His behest.  It also seems that 41% of Australians believe in astrology, maintaining that the position of various celestial bodies has a direct impact on your personality, the likely course of your life and, if you get right down to it, the entire future of humanity and our planet.  But my personal favourite is that 22% if Australians believe in witches, shadowy figures using sorcery and magic to demonise the unsuspecting.

So next time you use a focus group to provide an informed and rational ‘voice of the consumer’ on a significant marketing issue, bear in mind that of the six people in the room, three very likely believe that angels walk among us, at least two are concerned with the detrimental impact of Saturn journeying through their sign (but comforted by the Sun’s imminent presence in their compassion zone) and one person believes that a shadowy figure in an unfortunate hat is galvanising dark forces to challenge the natural order through the casting of spells and placing of curses.

But don’t let that impact your view of their credibility.  I’m sure their views on your advertising are completely logical and rational. They are, after all, your neighbours.

Filed under Advertising, Marketing, Research Tagged with ,

Unilever goes creative crowdsourcing

September 18, 2009 Leave a comment

I’m not sure I’ve ever read a more depressing advertising story. I want to be outraged, but the problem is that it’s just so overwhelmingly logical from Unilever’s perspective.

Unilever has announced that it will use IdeaBounty to generate its next campaign for Peperami. It’s fired its agency, Lowe London, because according to Matt Burgess, managing director of the Unilever division that owns Peperami, “Lowe has done great work on the account over the years. They’ve created a strong creative vehicle that’s extremely well-defined and very portable. But their great work has created a problem for them, because it makes Peperami the obvious candidate for crowdsourcing.”

The agency’s done such a good job, they’re not needed any more. I want to be outraged, but they’re so obviously right.

Because we do exactly the same thing ourselves. How many times has an agency used a senior team to crack a ‘big’ idea, produce the first couple of executions that establish the framework, then handed it on to juniors to roll out? We do it all the time, and we do it proudly. It’s responsible business and efficient use of resource. And so, unfortunately, it is for Unilever.

There’s more.

“We want to get the creative back from ‘good’ to ‘outstanding’ again. The best way to increase our chances was to increase the amount of creatives exposed to this brief. This is the overriding driver.”

Again, I want to be outraged. The problem is that there aren’t enough creatives exposed to the brief? Surely only a client could think quantity more important than quality when applying creative minds. But, once again, we do it ourselves. How many times have we stood before a client in a pitch, or even just a significant presentation, and told them that this is such an important/interesting/last-ditch project that we’ve put ‘all the agency’s resources to work on it’? We say it all the time, believe it to be true, and so teach clients that more is merrier.

But we cling to the belief that it doesn’t happen in other ‘professional’ industries. We like to point out that it wouldn’t happen with doctors. You might get a second opinion, maybe, but the key to arriving at a diagnosis isn’t just throwing more doctors at the problem. Which surely proves that the focused eye of experienced experts is much more valuable than just randomly having more people look at it (whether it be a medical or an advertising problem)?

But it’s a different issue, because when diagnosing an illness you want the right answer, of which there is only one (and, importantly, one that can be proven to be right). But when you’re developing a creative idea you want the most interesting answer, of which there will almost certainly be many (none of which can be proven to be right). So the parallel is flawed.

But the real issue is time and cost. The reason you wouldn’t get a dozen doctors to diagnose the cause of your sore throat and shortness of breath is that it would take too long and cost too much. You’re in some discomfort, you want someone to tell you why and give you something to make it stop. So you go to a doctor you trust, pay them for their expertise and do what they tell you because it’s reasonably immediate and reasonably affordable.

But if it were practical to get twelve doctors in one room, all listening to you describe your symptoms, all being presented with your vitals and then all giving a diagnosis, well that would work. You’d get access to a broad range of experience, a good chance of someone having dealt with your issue before and possibly even the confidence of seeing that the majority of doctors are all of a similar mind. And getting that diagnosis wouldn’t take any longer than visiting your regular GP. Getting treatment started wouldn’t take any longer either. And if as a bonus you only had to pay a fraction of what you currently pay your GP, then that’s a win/win/win.

And they’re the two barriers that crowdsourcing services remove – time and cost. You’ve got the same timeframe, reduced cost and more ideas. How could a Unilever not see this is a good thing?

We also throw up the argument that to develop great work you need people who are intimately familiar with the brand, and to whom the brand’s language is second-nature. This closeness, we argue, is what enables these people to deliver consistently outstanding work. But then in a completely contradictory stance we argue that the essence of any great brand should be able to be distilled to one word, any great idea captured in a simple statement and any decent brief distilled to a single-page. Which really means that any strong creative mind should be able to develop good work for such a brand. So again, you can see the client’s logic for a crowdsourced solution. They’ve defined the brand, established the campaign idea and have a clear brief for the execution. You don’t need familiarity with the brand, which means you don’t need an ongoing relationship with a good agency.

I also don’t believe we can claim any great surprise. Clients have been building towards this for a long time. I’ve been in the industry 20 years and most marketing departments have been crowdsourcing for at least that long.

Because for most clients, crowdsourcing is what focus groups are for. Put eight people in a room, give them a half-baked new product idea and ask them to redesign it. Then slavishly do whatever they suggest because it’s ‘what consumers want’. Change the format, the flavour or the font. Ask them what the packaging should look like, what cause the brand should support and who should be cast in the ads. Ignore the opinions of the experienced professionals and take guidance sourced from the interested amateurs. That’s crowdsourcing.

And unfortunately, this may give us the strongest clue as to how this will evolve. I’d suggest that just like the research industry, we have two choices available to us. We either become ‘coordinators’ or ‘sages’.

The ‘coordinators’ will basically be agency structures that provide a framework for producing stuff. Help establish the task, source a few ideas, get told what to produce and get on with it, much like the research companies that do little more than book the room, make sure the sausage rolls are warm, put the consideration in the envelopes, summarise the verbatims, bind the document and find the amusing clip art for the powerpoint presentation.

The ‘sages’ will be the agencies (or individuals) that help clients make sense of complexity and deliver ideas that generate great leaps forward. These agencies will be at the heart of defining the problem, identifying the opportunity and create the work that changes fortunes, much like the researchers who help clients makes sense of the world, delivering clarity, insight and actionable opportunity.

And of course the ‘sages’ will have value and get paid adequately. While the ‘coordinators’ will be commoditised and have to fight for every dollar.

Which unfortunately, once again must make perfect sense in the Unilever world. As a client, why wouldn’t I choose between these two options as the situation warrants? I want to be angry and rail against the injustice, just as I want to be able to mount an argument that suggests the whole argument for creative crowdsourcing is flawed. But I just don’t think that from a client perspective it is.

I don’t believe that crowdsourcing is likely to deliver a great, fortune-changing idea for a brand. I do believe that working closely with a collection of smart people (possibly within an agency) might.  Because that’s been my experience.  But I just can’t find a way to explain to a company like Unilever why that’s the case.

Filed under Advertising, Marketing, Research Tagged with , , ,

Psychographic Profiles – Can you help?

September 14, 2009 1 Comment

I went to a research presentation the other day.  It shared the findings of a reasonably detailed qualitative exercise looking at refining a potential target audience for a new product launch.

There was nothing wrong with the research itself. It seemed pretty robust, gave a useful insight into the way in which people might use the product, and actually quite an interesting observation on an unexpected occasion for which the product might be relevant.

The bit that I found interesting/frustrating was the audience description.  It was summarised into a psychographic profile of the audience, detailing those key behavioural traits that distinguish them from the population at large. The researcher presented it with due enthusiasm, the client spoke about it as a valuable springboard for creative and media development, and the agencies (creative and media) looked at each other in a slightly bemused fashion.  It all kind of made sense, but there was just nothing meaningful in it.  They are the kind of descriptions that seem to mean something until you look at them a little more closely (much like the lyrics of Coldplay, the books of Deepak Chopra or interviews with Bono).  They are of the ‘enjoys foods from other countries’, ‘ family and career are both important’ and ‘enthusiastic about technology’ variety.  Which don’t really represent a springboard at all. They don’t tell you anything of substance, and they certainly don’t distinguish people in such a way that you feel you could write specifically for that audience. They’re so broadly true, so completely generic, as to be meaningless.

So I want to start a running list of useful psychographic descriptors, the things that actually describe a meaningful trait that brings a person to life.

I start with ‘men who wear Oakley sunglasses with a suit’.   And, “people who use the word ‘nifty’’

Any thoughts?

Filed under Advertising, Marketing, Research

This is why I find research frustrating.

August 25, 2009 1 Comment

I might as well just state my case upfront.  I find most of the marketing research I’m exposed to massively frustrating.  It’s not because I don’t value research or that I see it as a handbrake on the creative process. My issue is that in most cases what I see is research that tries too hard to deliver significant insight.

The logic is simple.  As a marketer you undertake research with the goal of uncovering something substantial – something that delivers competitive advantage and value. No researcher wants to complete a research project and report that they didn’t uncover anything particularly interesting.  So no researcher ever does.

Those commissioning the research are looking for ‘insight’. But what if the research doesn’t yield an insight? The simple solution is to find a mildly interesting learning from the research and draw a grand conclusion from it. Which is why the majority of research ‘insights’ are nothing more than churched-up observations.

Which is what got me interested in a story headlined – “No comfort in comfort foods during tough economic times”.

On the surface it looks interesting, but closer investigation suggests that the sweeping, generalised conclusions being reported bear very little resemblance to the substance of the research.

This research supposedly demonstrates that, contrary to popular belief, during times of upheaval we don’t gravitate to the old and familiar, but instead show increased propensity to experiment and try new things.  I thought this was interesting. It certainly runs counter to my own experience.

It was referred to as the ‘comfort food fallacy’ effect. I imagined the research might highlight that during a period of significant personal upheaval (such as a goodly proportion of the western world is currently experiencing) we might find ourselves dismissing Macaroni Cheese like mother used to make in favour of, say, Black Cod with Fennel Chowder and Smoked Oyster Panzanella.

As a marketer I would find that interesting. More importantly, I would find that useful, the kind of insight that might guide my decision making and lead me down a path of new product development and marketing focus.

But that’s not what the research revealed at all. What the research actually tested was this:

‘In a prediction study, participants were told about a person who was described either as being in a very stable life situation or in the midst of many changes. Participants were then asked to predict whether this person would choose either a highly familiar or unfamiliar version of a similar snack (a very popular and well-liked American potato chip in familiar flavors or an unknown British potato “crisp” in exotic flavors like Camembert and Plum). Participants predicted that the stable person would choose the exotic unfamiliar crisp and the person in a state of change would choose the familiar chip. They explained their predictions by saying that the stable person would have more time and energy to try new things and the person experiencing change would be more interested in choosing a known or “sure thing” option. However, in a separate choice study, participants were asked to rate the level of change and upheaval in their own lives and then, in a later task, given the opportunity to choose either the familiar American chip or the unfamiliar British crisp. Opposite to the predictions, participants who were experiencing more change were less likely to choose the old familiar favorite and more likely to choose the new and unfamiliar option.’

The important point seems to be that they’re still eating chips.  To choose a camembert-flavoured chip over a potato chip does not suggest to me an overwhelming interest in experimentation and trying new things.  It certainly doesn’t suggest anything like a move away from comfort food or the extrapolated conclusion that during these challenging times there exists an unexpected societal embrace of significant behavioural change.  It doesn’t make a fallacy of the belief that people like comfort food in uncomfortable times.  These people are still eating chips and chips are still comfort food whether plain, plum or placenta flavoured.

(As a sideline, I was also amused that participants explained their prediction that the stable person would choose the exotic and unfamiliar crisp by noting that ‘the stable person would have more time and energy to try new things’.  Now, chips may be many things (tasty, moreish and devilishly bad for you being three) but they’re not really time-consuming or physically draining, are they?  I have never turned down an opportunity to sample a new chip on the basis that I simply couldn’t find a window in my diary or summon the physical reserves to attack a handful.)

The research went on.

“Wood’s study went beyond comfort foods and looked at “familiar anything.” When individuals in her study were in more upheaval, they were more likely to download an unfamiliar song or jog in a new park”.

They’re not exactly dramatic changes in behaviour, are they? Not really significant leaps into the unfamiliar. The same people are still listening to music and they’re still jogging.  The upheaval in their lives hasn’t caused them to swap their interest in music for weekend fox-hunting, or exchange their regular jog for evening crunk classes.

Again, my problem isn’t actually with the research.  It’s with the headlines used to summarise the research, the insight supposedly uncovered, and the conclusion therefore drawn.

“No comfort in comfort foods during tough economic times, study finds”

“Common connection between comfort food and crisis debunked”

“Comfort foods are not too comforting in poor economy”

Those conclusions are massive, completely unrealistic leaps from the evidence at hand. “Common connection between comfort food and crisis debunked” is the headline, but it’s not the reality.  They were still eating chips. Nothing was debunked.

My frustration is that you can just see how this unfolds.  Somewhere a marketing director is briefing her team. The product development team are being told to develop a product that reflects the average person’s desire to be much more experimental in times of hardship or stress (which was, of course, the insight from the research).  Then a brand manager will be told to develop a launch programme for this product, remembering that the research told us to focus on the fact that people don’t want comfort food during difficult times, but instead want to experiment.  Then an agency will get a brief to develop a campaign focused on the proven desire of people to embrace new things as a direct reaction to challenging times.

All of which is a massive leap from what the research actually highlighted, doomed to failure, likely to cost the business a lot of money, will probably get the agency fired, and all, absurdly, the direct result of someone choosing a camembert flavoured chip when in the presence of a researcher searching desperately for an insight.

And that’s why I find research frustrating.

Filed under Advertising, Marketing, Research Tagged with , , , , ,

Follow

Get every new post delivered to your Inbox.

Join 158 other followers