An Elephant In Our Room

I first caught wind of marketing research (MR) in the late ’70s when a college roommate, an MBA student, began watching TV commercials intently and frequently commenting out loud about them. When I inquired about this sudden interest in commercials, he mentioned he was considering marketing research as a career.  I’d heard of the “Nielsen Ratings” but not marketing research, and when he explained what it was I was fascinated because in many respects it resembled my own area of study, Institutional Research.

One thing led to another and my first “real job” was a marketing research position in the financial services industry a few years later. This job, as well as work I’d done as a graduate research intern, also introduced me to what is now known as data science.

Many things have changed since then!  Marketing is now vastly more complex. Globalization has actually happened. In the world of analytics, Bayesian statistics, latent class and a wide range of data mining tools have taken root, and many methods for time-series analysis and longitudinal modeling that were new at the time are now well-established in statistics, econometrics and other disciplines. The same can be said of categorical data analysis, which comes in very handy for those who work with consumer survey data. Statisticians now have an assortment of techniques to draw upon that few could have imagined when I was starting out.

Mainframes – then the dominant mode of computing – are now scarce and IT and other computer science footprints are everywhere in our daily lives as well as in MR. Data collection is no longer restricted to mall and street intercepts, RDD and mail (postal) surveys. There are many more options on the qualitative front as well! In a nutshell, there is a lot more to learn now if you want to become a marketing researcher and vastly more if you aspire to work in marketing science than when I started out.

This is a two-edged sword – we can now do much more but have to learn much more to be able to do it! Fortunately, we have online courses and webinars, F2F seminars, and specialized books and publications can be ordered over the internet from just about any place on Earth. University programs in marketing research are still rare, though, and most marketing researchers I know have entered the field with little or no university background in research methods or statistics. The foundations of research are not being instilled and automation – including the ever-elusive “AI” – cannot make this go away. Automating bad practice doesn’t make it good practice. It just makes it more cost effective…

By contrast, years ago the big MR agencies, though small by today’s standards, were generally very good places to learn the fundamentals of marketing research. Beside OJT there was a substantial amount of in-house training and junior staff were also sent to seminars once or twice a year. Many boutiques had been founded by people with strong academic backgrounds in research and also were excellent learning environments. Now both the big agencies and many of the boutiques, as well as mid-sized agencies, are heavily productized – essentially software venders in some cases. As a result, most MR veterans I’m in touch with feel the skill level in MR has been declining, at least in the economically developed countries. (Thirty years ago, there was little MR in most of the world so there was no place to go but up, so to speak.)

Though the difference is not like night and day, when I began my career the fundamentals of research were better grasped by the “average” marketing researcher. The fact that these tools were developed many decades ago is entirely irrelevant – the foundations of science never grow old. Though I would not like to go back to the days when “IT” was a new concept, in some ways doing things the hard way made it easier. The consequences of mistakes made early in the research process loomed larger later on because they were difficult or sometimes impossible to fix, and this forced us to think much more carefully about what we were doing and why. Tab plans are one example.

Now, though, we seem to spend an awful lot of time fixing mistakes that could easily have been avoided with more thinking upfront, and one result is that total time needed for many projects is just as long now as it was 30 years ago! Conducting serious business by Smartphone isn’t always very smart either, and serious miscommunications are commonplace.

There are also many programmers who know little about marketing or research running predictive analytics for various purposes. Correlation is confused with causation and all sorts of other elementary statistical errors are commonplace. Though point-and-click software is often blamed, truth be told, it doesn’t take much skill to copy/paste open-source software commands from manuals, make small modifications and press enter.

Much closer to home for most of you, DIY questionnaire software is increasingly faulted for what many (including me) see as a decline in the quality of consumer surveys. My own impression is that it’s not the software to blame as much as the poorly-trained humans using it. Again, the basics are missing. It’s not just quant, as many of my contacts who specialize in qualitative research will remind me. There almost seems to be the idea that if you are able to speak your native tongue, you can do qualitative research.

Likewise, sampling is not something people in MR think about as much anymore and this affects both quant and qual. Knowledge of sampling is crucial in any kind of research. The same can be said for research design, and fewer seem to be aware of the various types of experimental, non-experimental and quasi-experimental designs and how they relate to causal inference. That is not progress.

There are lies, damned lies, and then there are data visualizations…I recall a colleague commenting that many marketing researchers struggled to read cross tabs because they had been “raised on PowerPoint.” She made this comment more than 15 years ago and since then numeracy has declined, in my opinion.

Both qual and quant reports are sometimes little more than collages with lots of buzz and client jargon printed on them. These reports look good, but many clients are still complaining and wondering where our recommendations (if even offered) came from. Reports often lack substance, in other words and, worse yet, fact and speculation are frequently conflated. At the other extreme are piles of PowerPoint charts, which are costly to produce and satisfy no one except perhaps the most OCD clients. Though I still see highly impressive work, critical thinking skills – indispensable in any research – seem to be on the wane in MR. Presentation skills have improved, IMO, but this too is a double-edged sword, for reasons that should be obvious.

To be fair, I should not point the finger only at marketing research agencies. Even at large multinational clients the status of marketing remains low and not many have well-funded, highly-skilled marketing research or consumer insights departments. There a chicken-and-egg aspect to much of this.

So, to sum up, The Good Old Days never existed. We now have more tools and better tools than ever before. We have more outstanding researchers than ever before – in terms of headcount. As a proportion, however, their presence has diminished because MR has grown so rapidly, and they have less influence than before. Training has not kept pace with demand. We also have many people from other industries who know little about marketing or research now doing marketing research, and doing it badly.

I should note that, though all my work is now customized, I’ve been a member of two R&D teams that developed standardized proprietary methodologies and feel strongly that these methods have their place. Some have required highly-specialized expertise, considerable investment and many years to develop and cannot be matched by ad hoc methods. Nevertheless, a substantial amount of marketing research will always need to be tailored to the specific needs of a specific client at a specific time. If we lose our customized research skills, “marketing researchers” will have become little more than salespeople moving product, and both our industry and clients will suffer. Moreover, we will lose the knowledge base needed to develop new proprietary methods and automated tools that are really needed and that really work.

Dave McCaughan and I began our MR Realities podcast series in February 2016 and its sole purpose is education. We began the series because we both found ourselves spending more and more time explaining MR basics to other marketing researchers, and this quickly becomes very costly. It is also nerve-wracking, to be honest. While our experience and that of other veterans we know might be unrepresentative, I think that’s a long shot. It’s more likely that the quality of MR is in decline even if some of us haven’t noticed the elephant.


This article was inspired in part by discussions with Bob Lederer of RFL Communications.

Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy. He has more than 30 years experience working in marketing research.

Arrange a Conversation 


Article by channel:

Read more articles tagged: Analytics, Data Visualisation

Data & Analytics