The Dangers of Market Research Interpretation

 

Remember the Pepsi Challenge?  How about New Coke? 

In 1972, Coca Cola had been the dominant soft drinks company in the US market for most of the 20th Century with 18% of soft drink users describing themselves as exclusive Coke drinkers, compared with 4% for Pepsi.  By the early 1980s, Coke had dropped to 12% whilst Pepsi had risen to 11% - despite the fact that Coke was far more widely available and spent over $100m more on advertising each year.  Coke were extremely worried.

Against this backdrop Pepsi were running the Pepsi Challenge: a blind taste test where randomly selected participants were asked to take a sip of each cola and state their preference – Pepsi won nearly 60% of the time.  The Coca Cola executives ran a number of market research exercises of their own and they seemed to confirm Pepsi’s results.  Customers described Pepsi as “rounded” and “smooth” and appeared to find Coke too harsh.  Coca Cola felt the research showed that customer’ tastes were changing - This signalled the start of the New Coke project.

Coca Cola scientists went away and played with the formula of Coke to make it lighter and sweeter, with less bite – in other words more like Pepsi.  They ran taste tests on the new formulation and saw a marked improvement against Pepsi.  They continued to refine the formula on the basis of their testing and eventually had a product able to beat Pepsi in blind taste tests nearly 60% of the time.  This product was launched as New Coke and the Coca Cola CEO described the launch as “the surest move we have ever made”.  It bombed – Why?

The Pepsi challenge is a sip test for a reason.  After taking just a sip of two drinks, the majority of people will state a preference for the sweeter of the two.  However, of you ask the same people to drink the entire can, or to take a number of cans of each drink away with them, drink them over the course of a few days or weeks and then report back you get a very different result.  The sweetness which was so appealing in a sip test becomes cloying and when questioned the testers will generally chose Coke around 60% of the time.

The Pepsi challenge was a clever marketing ploy and in some senses a distraction.  New Coke was dropped and the perceived threat of Pepsi never materialised – Coca Cola remains the largest soft drinks company on the planet by a considerable margin. 

So what’s the point of this story?  Well perhaps there are lessons to be learnt for us all:

 

• To be careful with the questions that we ask when conducting research – the way that such tests are set up will directly affect the feedback we get.  The Pepsi challenge was not a fraud or wrong in any sense, but the interpretation of the results was wrong.  All that the results really showed was that people react differently to different colas in different situations. 

 

• To be careful with the decisions we make based on market research.  Information gathered in this way can be absolutely invaluable, but when it has the appearance of scientific fact it is all too easy to ignore other solid facts in the face of seemingly incontrovertible “evidence”.

 

• To be wary of testing in artificial conditions.  In the case of the Pepsi challenge the test was artificial both because it was a sip test and because it was a blind test.  In reality, people almost never drink just a sip of their drink and they do not taste their drinks blind.  The test ignored the fact that packaging, branding and so on have an important effect on both people’s drinks choices and their subsequent enjoyment of them.  This is particularly important in catering where a host of environmental factors affect the customer’s experience in addition to the quality of the produce they receive (service, packaging, ambiance, comfort, cleanliness, promotions and so on).

 

• To be aware that people find it very hard to explain their preferences – they often lack the vocabulary of a professional taster, or in our case a professional caterer, to verbalise why they “like” a product or a venue.

 

• To be wary of “throwing away the baby with the bathwater”.  At the time of the Pepsi Challenge, Coca Cola were the dominant player in the marketplace with the largest dedicated following.  In chasing another demographic they risked alienating their core user group – the resultant backlash was huge. 

Coca Cola reverted to “Classic Coke” very quickly and wrote the New Coke experiment off as an expensive mistake.  How many companies would be able to swallow a loss of that scale though?

For a far more detailed exploration of this episode and many others relating to rapid decision making we recommend reading Malcolm Gladwell’s book, Blink.