Random Thoughts on Marketing Research, Statistics and Data Science (June 12, 2018)

My opinions. Nothing more. What real evidence do we have regarding the effectiveness our organization’s marketing? Beliefs are not evidence, however logical they may seem to us, and however many of us subscribe to them.

I’ve heard it claimed that the use of neuromarketing methods in choice modeling (“conjoint”) results in dramatic improvements in correlations with actual sales. This is odd, since choice modeling is normally used to evaluate hypothetical products, thus no correlation with actual sales is possible.

There are also pricing studies in which respondents are shown descriptions of real products in the choice tasks. In these studies, the rank order of preference share for the brands at average prices, and the rank order of actual sales around the time of the study for these brands should be nearly identical. This is a standard diagnostic. The correlation between share of preference at average prices and actual market share should also be high.

Thus, if there is substantial room for improvement in these diagnostics, it suggests deficiencies in the modeling or perhaps fieldwork issues. Sales and share data also can be problematic, though, and are not the Word of God. They also are unavailable for many product categories.

Neuro marketing is not all baloney but, with so many competing claims, it can be hard to tell which ones to believe. We shouldn’t be afraid to ask tough questions. 

Marketing researchers can waste a lot of time and valuable budget re-inventing the wheel. To make matters worse, the wheel they re-invent may quickly fall off.

As a former R&D person for Nielsen Customized and Kantar, I’m very mindful of this. We did our homework. Over the years, I’ve found The Public Opinion Quarterly, The Journal of Marketing Research, The Journal of Marketing and The Journal of International Marketing very helpful. There are others I do not currently subscribe to but keep my eye on. Three are The Journal of Consumer Behaviour, The Journal of Consumer Research and The International Journal of Market Research.

There are also peer-reviewed journals on statistics and machine learning, such as The Journal of the American Statistical Association and The Journal of Business & Economic Statistics, that are excellent resources for marketing scientists.

Missing Links may also be of interest to many of you.

“Science is what you know, philosophy is what you don’t know.” – Bertrand Russell

One could also argue that part of science is determining what you don’t know and learning about it. Philosophy is what we lean on when science fails.

Whatever the case, one thing I’ve learned is not to debate philosophers. 😊

I often hear people say the way a story is told is just as important as the story itself. Obviously, this makes sense for fiction, but for marketing research? Think about it.

To be sure, important findings can get lost in a muddled presentation, but a captivating presentation may be slipshod research extravagantly dressed. An inaccurate story well told is not an accurate story, and best ignored by decision makers.

Isn’t our first obligation to get our research right? Primum non nocere.

Despite serious criticisms from marketers and methodologists, NPS is well-established in many companies. Ironically, it also seems underutilized.

Though NPS is an aggregate index, the recommendation question on which it’s based is not, and can be modeled in all sorts of ways to shed light on what is driving movement in the ratings over time, or differences in the ratings among customers.

Arrange a Conversation