Combining Smart Design with Smart Analytics

Historically, there has been something akin to a Great Divide in the minds of many marketing researchers: use focus groups or in-depth interviews to uncover insights and consumer surveys to get the numbers.

While there are now many other data sources and methodologies we can make use of, tying them together and making sense of them is another matter. Sufficient expertise may not be available in-house. Also, a lot of data are messy and unusable without substantial cleaning and editing. What’s more, each data source only reveals part of the story and may tell a story that contradicts other data. Though I’ve long been an advocate of data integration, I will admit it’s not always easy.

Fortunately, there is a simpler alternative. Though it’s become fashionable to trash survey research, it’s actually one of the most valuable methods we have and often will give us much of what we need. Unfortunately, there seem to be fewer and fewer researchers who excel at it and one predictable result, besides an abundance of substandard surveys, is that some criticisms of the methodology are downright silly, with a few straw men thrown into the mix. One could almost come away with the impression that only two types of consumer surveys are possible: 1) very short surveys designed to get a quick read on a single issue; and 2) mammoth tracking studies that consume a lot of time and budget just to tell us little has changed since the last wave.

It’s certainly true that there are lots of very short surveys and many which are excessively long. Either can be of dubious worth. Something that is frequently overlooked, however, is that in both cases the “analytics” nearly always consists of simple crosstabs and graphics or, perhaps some correspondence analysis maps run by a junior research exec or even a production person. Typically, in large-scale tracking and usage & attitude studies a huge amount of data goes to waste, and conclusions and recommendations, as a consequence, are often merely descriptive summaries. Small wonder many clients complain!

There is of course much more to survey research – and quantitative research generally – than these two kinds of surveys. There are conjoint studies and many sorts of proprietary methodologies, to name two examples. What is often forgotten is that quantitative research – surveys in particular – can also be used “qualitatively” to help marketers unearth important insights about consumers. Quantitative research, including Big Data analytics, does not have to be confirmatory – it can also be exploratory and is much more than what percent said this and what percent said that. Segmentation and key driver analysis, for example, are excellent ways to learn about consumers, irrespective of how the results are used.

I should stress that I am not advocating data dredging, an example of which would be running a massive number of crosstabs and cherry-picking results we like! This all-too-common practice is quite risky since the chance of a fluke result increases with the number crosstabs we run…and we may not be so lucky the next time around. Another danger is that looking at variables two-at-a-time can be very misleading. For example, it may appear that older consumers are heavier users of a particular category but, after taking gender, income and other characteristics into account, we may find that category usage actually declines with age.

Even attitudinal studies that, on the surface, appear sophisticated frequently include scales that have been improvised or are legacy items with questionable validity. Modern psychometric tools are seldom used in marketing research to adjust for respondent scale usage patterns and background characteristics. In segmentation studies, the old factor-cluster “tandem approach“, which came under attack by leading academics more than 20 years ago, still enjoys widespread use. Small wonder many clients complain!

I should also be clear that I do not believe surveys by themselves will always give clients everything they need to make good decisions. That will seldom be the case and, in fact, other information sources such as focus groups, social media, transactional data bases and market share figures normally must be leveraged to design a good survey and interpret its results. The importance of understanding the client’s business and their research objectives also cannot be overstated. Nor would I downplay the role of new and radically different methodologies. Nonetheless, in my opinion, some marketing researchers do not fully appreciate the insights customized consumer surveys can provide.

Besides a tremendous variety of data, we now have vastly more ways than ever to analyze data of all kinds, including survey data. Some advanced analytics tools are actually not brand new but have yet to diffuse very far into marketing research. One reason is that there are so many of them! Another is that they usually work best when designed into the research in a way that allows flexibility to respond to the surprises data enjoy throwing our way.

Marketing research is becoming increasingly specialized and the requisite know-how is lacking in many MR agencies and client-side MR units, even when there is broad familiarity with some of these newer techniques. Making advanced analytics work involves much more than math and programming. Last but not least, many agency-side marketing researchers are under pressure to sell standardized proprietary methodologies with high profit margins and discouraged from using customized methodologies. But are clients looking for competent salespeople or competent marketing researchers?

Combining smart research design with smart analytics can go a long way to help our clients make better decisions and make them more efficiently. Marketing research needs to re-discover research. I’ll have more to say about these topics in future posts and gave a quick overview of analytics at the Festival of the NewMR February 2nd, 2016.

Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy.

Arrange a Conversation