The Qualitative Side of Quantitative Research

Kevin and Koen may buy the same brand for the same reasons. On the other hand, they may buy the same brand for different reasons, or buy different brands for the same reasons, or even different brands for different reasons. The brands they purchase and the reasons why may vary by occasion, too.  

What is quantitative research?

Quantitative research has been defined in various ways. Here is one definition:

Quantitative methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through polls, questionnaires, and surveys, or by manipulating pre-existing statistical data using computational techniques. Quantitative research focuses on gathering numerical data and generalizing it across groups of people or to predict or explain a particular phenomenon.

In marketing research, “quant” historically has meant consumer surveys. Analysis of consumer survey data has typically been limited to reporting numbers, perhaps broken down by age group, gender and a few other respondent groups of interest. The emphasis is mainly on the Who, What, When, Where, and How, though segmentation, conjoint, key driver and other analyses that delve into the Why are also occasionally conducted with consumer survey data. Marketing mix modeling and predictive analytics are two forms of quant that do not require consumer surveys, and usually draw upon other data sources such as sales data. Survey data may be integrated with these data or used for reference, however. 

By and large, detailed exploration of the motivations underlying the behavior of consumers has been left to qualitative methods such as focus groups, in-depth interviews or, more recently, insights communities and social media analytics. This contrasts with other disciplines, such as psychology, where quantitative research is frequently the primary tool used to understand possible causes of behavior. Note the last four words in the definition of quantitative research given earlier.

We feel quant is used too narrowly in marketing research, especially in light of today’s fast computers and rich array of statistical methods for analyzing data. Owing to small sample sizes and the high measurement error associated with coding verbatim comments, qualitative research can suggest hypotheses but cannot explore them beyond a certain point. Quantitative methods can also be used to develop hypotheses, not simply to test them. Many marketing researchers seem to have forgotten this in the rush to embrace new technologies which may or may not actually be useful.

Attitudinal measurement

In recent years, an urban legend regarding attitudinal measurement seems to have surfaced, namely that attitudes can only be measured by traditional 5-point agree-disagree scales and that this does not work. MaxDiff is sometimes offered as a solution though, often, the value of attitudinal measurement to marketers itself is questioned.

Actually, there are many kinds of attitudes relevant to marketers that can be measured in several ways, 5-point agree-disagree scales and MaxDiff being just two. Attitudinal measurement is utilized extensively by scholars in many disciplines, including marketing. For examples, see the Journal of Marketing Research, the Journal of Marketing and Journal of International Marketing. Textbooks have been written on this subject, and two popular ones are Marketing Scales Handbook (Bruner) and Handbook of Marketing Scales (Bearden et al.). Importantly, geo-demographics and life stage usually only partly explain these attitudes.[1]

What is generally true, however, is that fundamental values, such as religious beliefs and political orientation, are not useful for marketers in most product and service categories. Attitudes relating to variety-seeking, hedonism, materialism, uncertainty avoidance, price sensitivity, locus-of-of-control, and country of origin, to name just a few, are highly pertinent to many categories. This link will take you to literally thousands of scales that have been used by marketing scholars and marketing researchers.

The relative importance of various aspects of functionality, such as ease of use and cleans well, can also shed light on consumer behavior and suggest clues for new product development.[2] Furthermore, utilities derived from conjoint analysis and MaxDiff can utilized in various kinds of multivariate analysis, not just ordinary segmentation.

Attitudes, behaviors (claimed behaviors in the case of surveys), and demographics can be tied together in segmentation and key driver analysis in ways that were not possible until recently. Structural Equation Mixture Modeling (SEMM), for example, combines cluster analysis, factor analysis and regression.[3] There are many other recently-developed procedures, in addition to older methods that have not diffused into marketing research.

Consumer surveys are much more than simple tracking and the ill-timed customer satisfaction surveys that annoy us when our credit card company has fouled up again. Much of what we need to better understand why consumers do what they do and what they want, is already collected in many usage and attitude and tracking studies. However, as noted earlier, these data frequently are merely reported and not analyzed beyond simple cross tabs. Savvy marketing research veterans have long known, however, that what is contained in Management Summaries of qualitative research can often be tested and explored in more depth in a follow-up quant study designed for this purpose. We can now do this faster, more cheaply and better than ever.

Consumer surveys, of course, are just one kind of quantitative research. But they are data rich and, furthermore, primary research has many advantages compared to analyzing data that have been collected for other reasons.[4] Designed and conducted appropriately, consumer surveys can tie together the Who, What, When, Where, How and Why. This is very useful for new product development, and for communicating with consumers; Kevin and Koen may buy the same brand, but for different reasons, and the same ad or offer may work for one of them but not the other. 

Moreover, customers can buy most products from a wide assortment of outlets. Many use cash and are not members of loyalty programs. There are other limitations as well. A bank, for instance, will have data on its customers from a certain period, but not prior to that, and no data about its customers’ transactions with other banks. Moreover, light users are an important source of volume in most FMCG categories because there are so many of them. Data on light users will necessarily be very sparse and will be non-existent for people who do not use a category, a critical group for new varieties of products. Lastly, even “hard” data such as these are seldom error-free even when reasonably comprehensive.

Most secondary data are incomplete, but this does not mean they are never useful. Increasingly, marketing researchers are combining data from various sources, an example being a bank including a sample of its customers in a usage and attitudes survey of banking customers. The data the bank has in its customer records can help it interpret responses of its customers and, in some instances, those of the total sample. As a recent academic example, Ilhan and coauthors[5] identified Facebook users who attack a brand on its own page with qualitative research. Next, machine learning and time series analysis showed the prominence of such behavior, how it motivates brand fans to defend the brand, and to what extent this full chain of events hurts or benefits the brand’s engagement.

Some examples 

Here are a few examples of how quantitative research can help us better understand consumers, what they are seeking that we are not now offering them, and how to effectively communicate with them:

• A financial services company undertook a segmentation study among general consumers and a sample of its own customers. Internal customer data were leveraged to enrich the survey data and flesh out the segments, and to help management better understand the Why underlying the What for their own (and possibly competitor) customers. For example, one key finding was that risk acceptance/aversion, general financial sophistication and other attitudes predicted interest in new investment vehicle concepts beyond that explained by life stage, demographics and historical behavior. 
• An online retailer wanted to reallocate marketing across tens of options. Suspicious of last-click attribution, they commissioned Vector Autoregressive (VAR) modeling that considered long-term effects and interactions of not just bringing prospects to the retailer, but also of increasing check-out and revenue. They found a much higher revenue impact of content-integrated marketing actions (e.g. affiliates and price comparison sites) versus content-separated actions (e.g. emails, retargeting) and increased revenues 17% with the same overall marketing budget.
• A credit card company wanted to ascertain what certain consumer segments are seeking most, and a conjoint and segmentation study among general consumers was conducted. The results challenged some important assumptions long taken for granted, while also providing context for some earlier marketing research the company had commissioned.
•  A fast-moving consumer goods company wanted to adapt its global dashboard to reflect different performance effects across mature vs. emerging countries. A Hierarchical Linear Model showed that advertising awareness and brand love are key in the former, but consideration and word-of-mouth are key in the latter. 
• A software company conducted a user experience (UX) and satisfaction study among a sample of its customers. Key driver analysis was performed with an advanced variation of structural equation modeling (SEM) and the results used for new product development as well as revision of user manuals and customer online help.

Future directions

With the advent of “big data,” quantitative analytics is receiving more attention and buzz than ever. However, it is our impression that the vast majority of quantitative analytics is descriptive or predictive, and does not attempt to explain to decision-makers what it is describing or predicting. This appears to be true in marketing research as well, as mentioned earlier.

In some cases, this is inevitable given the limitations of the data at hand or the obscurity of the causal mechanism(s) generating the data. Sometimes, though, it reflects low-cost factory-style analytics and that many “data scientists” have only had superficial education in the research methods and statistics – let alone marketing – that can be used to get at the Why and link it to the Who, What, When, Where and How. Collecting and managing data, and running random forests will not do the trick.

We also sense many marketing researchers are feeling overwhelmed by the emergence of new data sources and sophisticated technology for gathering and analyzing data. There also has been an inordinate amount of hype about “big data” and “data science”, which has proven very distracting and confusing for many of us. All this said, we are confident that, with time, our future shock will diminish sufficiently to allow us to extract much more value from data than we now do.

Arrange a Conversation