Why Statistical Consulting is Hard

At least two things in life are certain: death and uncertainty. Taxes can be avoided or evaded, and not everyone is liable for them. Even the proverbial well-oiled machine can inexplicably break down without warning.

But first, what is the role of a statistician? Here are some of the things we do:

• Design research
• Design sampling plans and supervise data collection
• Make projections from samples to populations
• Quality assurance 
• Develop and test hypotheses, including complex causal models
• Analyze data
• Make predictions and forecasts
• Perform simulations
• Interpret data, models and forecasts, and make recommendations

This is quite a lot, even for an experienced statistician. What’s more, it’s not unusual for a statistician to be brought into a project suddenly without sufficient briefing and be asked, in effect, to save the game. I’m sometimes reminded of relief pitchers in baseball who have little time to warm up before getting on the mound with the bases loaded and no one out. We’re often under a lot of pressure.

Most of our clients, internal or external, have had little formal training in statistics beyond the very basics. Serious misunderstandings about data and analytics are more the rule than the exception, and the meaning of evidence is frequently unclear.

Moreover, the way the fundamentals of statistics are usually taught – by plugging numbers into formulas – can give the impression that’s all there is to it. It’s just calculations. Of course, to one extent or another, everyone realizes it’s not quite this simple, but how complex statistics is may not be readily apparent to most. An Analytics Toolbox gives a snapshot of the methods I use, and there is much more than what I’ve listed.

Because of these misperceptions about statistics, and the natural human tendency to look for quick, confident, yes-or-no answers, statisticians can seem wishy-washy. Frequently, though, we really don’t know the answer, only one or more possible answers.

Part of this gap in perspectives is because statisticians are trained think in terms of conditional probabilities, i.e., given B what is the probability of A? This is not natural for humans – we prefer to put things into mental boxes, preferably this one or that one.

Let’s not forget politics, both personal and organizational, either. Here are some tight spots we statisticians can find ourselves in:

• The client is high on a research method they’ve used for years, a legacy of a now-retired guru still revered in the company. However, we have discovered this method has serious flaws. How do we handle this situation tactfully? 
• The client has apparently misinterpreted research conducted by a marketing research agency. They have been misled, but we aren’t sure. During the course of discussions, it becomes clear that the client has made questionable decisions as a result of the mix-up. What do we do?
• One or more key decision makers doesn’t want to hear what we have to say. It could be that our advice runs counter to an initiative they have been championing or a decision that, unknown to us, has already been made. Perhaps it just “doesn’t feel right” to them. For any number of reasons, they may choose to ignore our recommendations. We may be interacting with them for the first time, which makes it harder to know how to respond.
• It’s not unusual for there to be rival factions on the client side or many parties involved, including ad agencies and external consultants. We will not be able to please everyone and, if not careful, may displease everyone.
• “We already knew that!” can be difficult to respond to. Perhaps we hadn’t done our homework or were misinformed by a colleague or member of the client team. Perhaps the client is just trying to save face. “But now you have evidence you were right!” is not a good reply, though I’ve heard this and similar responses recommended for situations such as these.

Technical details are usually difficult to explain in simple, layperson’s words, and anytime we simplify, we risk misleading our audience. One reason statistics textbooks are usually jam-packed with mathematics is that many statistical concepts cannot be explained concisely in words. Statisticians need to think several steps ahead and try to anticipate reactions to our explanations and how we should respond to those reactions.

Statistics is seldom as simple as 1-2-3, and experience and judgment will always be crucial to what we do. Often our answers to your questions are just our best guesses. While we may wish to be as confident and self-assured as the cool cat in the background photo, we cannot deny or hide from uncertainty. And we shouldn’t try to, since the value of statistics is that it is a systematic and proven means of dealing with uncertainty.

If there were no uncertainty, there would be no statisticians.

Arrange a Conversation