Designing Research

A researcher unable to design research is not a researcher. In Preaching About Primary Research I discuss the distinctions between primary and secondary research, and express my concerns that marketing researchers have been losing their knowledge of how to design and conduct primary studies.

Just as a quick recap, and again borrowing from Wikipedia:

Primary research involves the collection of original primary data. It is often undertaken after researchers have gained some insight into an issue by reviewing secondary research or by analyzing previously collected primary data. It can be accomplished through various methods, including questionnairesand telephone interviews in market research, or experiments and directobservations in the physical sciences, among others.

With so much data now available, the temptation is to mine whatever data we have in-house or can obtain quickly for answers management needs. This is not unreasonable and is part of my job. However, there are many risks to taking this approach.

One is that our data will be incomplete. Another is that it will contain serious errors we will not be able to detect until it is too late. A third issue is that the chances of fluke results are heightened when we mine data. Different data may also suggest different decisions and add to our confusion instead of reducing it. More fundamentally, mining existing data can force us to adapt our decisions to the data, rather than to determine what decisions we need to make, and then what data and analytics might be used to enhance these decisions. Our thinking is turned upside down, in other words.

While I won’t reiterate what I’ve already written elsewhere, I will mention that I am worried about a dangerous delusion taking root, namely that “AI” can somehow magically obtain, or even create, all the data it needs, and is able to find The Answer with minimal human instruction or supervision. There is now so much hype and nonsense about AI circulating in the blogosphere, and even “respectable” news media, that many people are confused about what AI is and what it can and cannot do. 

Few decision-makers, even at the highest levels, have had much training in research methods, statistics or computer science. If they are among the exceptions and have worked as a data scientist or statistician, most likely it wasn’t for very long and their skills are rusty and outdated. Even some of the best and brightest at C-level can be taken in by exaggerated claims or, more often, misunderstand what true AI experts are really saying. This is a highly specialized and technical field, after all, and jargon-laden.  

Let’s return to the beginning. Why do we do research of any sort? If we are scholars or scientists, we are attempting to contribute to and expand the body of knowledge of our discipline. If we are business people, our focus is narrower and more immediate: Research can help us make decisions that turn out to be good ones. It can help the bottom line.

Many business decisions are one-off and unrelated to other decisions, and secondary data (or primary analysis of secondary data) can be likened to using a sledgehammer when a scalpel is needed. Customized research – research tailored to a specific decision or set of related decisions – is often required. Customized research is primary research.

So how do we design primary research? First, we must clarify our objectives and key considerations:

• What decisions must be made?
• Who will make them?
• When will they make them?
• When will the decisions be implemented?
• How will they be implemented?
• By whom will they be implemented?

At this point, we can begin to think about the data we’ll require and, in general terms, about the methodology and analytics. Timing and budget can then be considered. The foregoing will be old hat to seasoned marketing researchers…who may point out that, in the real world of business, things are not quite so tidy. I know. Often discussions go around and around, some research is conducted and then it’s back to the drawing board.

To put some flesh on these bones, let’s imagine that we’re in the personal care business. Sales of an important brand have been flagging, and we must take some sort of corrective action. First, we should use what data we have or can get hold of reasonably easily to try to understand possible causes of our disappointing sales. This is secondary research and normally essential. The data may come from a variety of sources, including internal data, discussions with retailers, our company website, published data, newspaper articles, social media and past research. It may be that this will be enough to solve the problem to our satisfaction. It’s the economy, stupid!

More typically, however, at this point in the process we’ll have answered some questions but things we hadn’t thought of will have come to light. We may conclude primary research will be needed. In these situations, Usage and Attitude (U&A) studies are common, though there are other options.

U&A studies, which are called A&U or AA&U studies by some of my fellow Americans, are consumer surveys that cover a wide range of topics. Sometimes they are done qualitatively first, e.g., in focus groups or in-depth interviews, and then followed up in a second phase with quantitative research. Topics include:

• Brand awareness
• Advertising awareness
• Brand usage
• Usage habits
• Brand image
• Brand selection criteria
• Psychographics
• Media usage
• Demographics

They are normally long – often too long – which erodes data quality because respondents grow tired or become bored and lose interest. Questionnaire design is the subject of many books and journal articles, so I’ll need to set that aside here. The design of the research, including what sorts of consumers will be interviewed, sample size, whether it will be done online and many other decisions must be made at this juncture. Multi-country research is also a separate topic but suffice it to say that many countries are multi-cultural and multi-lingual and that this also must be taken into consideration when designing research.

Brand mapping, segmentation and key driver analysis are sometimes “designed into” U&A studies. This is important because advanced analytics usually works best when the data have been collected with specific kinds of analytics in mind. (At the risk of offending some readers, this concept is entirely foreign to many data scientists, who are often forced to work with data already available, albeit with cleaning, recoding and restructuring.) 

Some marketing researchers have worked mostly with or, in some cases, entirely with standardized proprietary methodologies. It may come as a surprise to them that even research such as U&As that is frequently conducted can only be partly standardized and templated. Some marketing research is extremely complex and customized research in general is difficult to automate. I touch on this subject in Why Customize? 

I’ll conclude by pointing out that U&A studies are just one kind of marketing research, and that it is increasingly common in customized research to integrate data from various sources, including secondary or in-house data. Garbage In Garbage Out still applies, but customized research does allow us to reduce this threat.

Secondary research can sometimes get the job done, but in an increasingly complex and fast-paced world, the need for primary research, if anything, is growing. Whether you call yourself a data scientist or a marketing researcher, if you know how to design primary studies you’ll have a leg up on the competition.

Arrange a Conversation