Why did analytics fail us in the 2016 U.S election?

We’re so sophisticated, aren’t we? We track, collect, analyze and do predictive modeling using data from every place we can find it. We have software with extraordinary power to identify insights and trends, to distill meaning from numbers, to understand sentiments expressed in textual information. We have lightning bolts shooting from our fingertips as we concoct algorithms that we know are scientifically, statistically, and mathematically sound. And we are so damn cocky about it sometimes, aren’t we?

My life has been spent convincing customers and prospects that my software will shine the light of truth on their business, whether I was selling BI software, data warehousing software, predictive analytics software, or any of the other supporting technologies needed to enable these magical abilities to understand the past, monitor the present, and predict the future.  And I truly believed it would. The same kind of software, by the way, that many of these political pundits used to predict the winner in this year’s presidential election.

So what the hell happened?

I mean, they had more polls than they knew what to do with.  They had social media. They had focus groups.  They had survey data, and on and on and on. They could slice and dice that data by state, gender, race, religion, age, issue, party, likely vs unlikely to vote, and a whole lot of other dimensions. And it was refreshed constantly. They had the latest technology and the brightest technical minds in the business, and they had the political pro’s, AKA Subject Matter Experts, and they STILL blew it.

In my opinion there was data they were missing and it was the kind of data that’s really hard to collect or even quantify. In fact, you may call it “gut feel” or “anecdotal”. It’s the kind of data that’s so unstructured it makes unstructured data seem structured. Maybe the umbrella term would be “Passion”. But it was only accessible indirectly and via strange and unmeasurable means. 

As an example, if a few of these data scientist types had left their laptops behind and taken a drive with a couple political pros across the upper mid-west they would have seen home-made “TRUMP – PENCE” signs in people’s front yards and on road sides. These were not the mass produced, slick blue and white signs you see here in the northeast now and again. They were made of brown cardboard or ply wood, or sometimes poster board. They were hand lettered and the verbiage was personal, often expressing a sentiment. They were nailed to a post, or a picket from a fence, or tacked to the back of an old chair. And they were everywhere. “So what?” you may ask. Well, from my perspective, if someone takes the time to make a sign and put it out unsolicited, then THAT’s someone who will not only vote on election day, they will persuade their friends and family to do the same. These folks were committed!

In the same way, the rallies Trump held were not only large, they were rowdy.  People screamed like they were at a rock concert. They pumped their fists furiously. They attacked others who didn’t share their enthusiasm, sometimes including other Republicans. Like it or not these folks were all in for Trump. They saw him as the answer to their woes.  He was going to return their lives to the good old days when things were dependable and familiar. They were safe and happy then, and this new world is odd and kind of scary. They were afraid of the terrorists, the economy, the illegal aliens, and the untraditional ideas of the Democratic Party which seemed hell bent on changing the world they felt most comfortable in. They felt Trump would save them. There was an intensity, a fervor, a passion. It wasn’t based on policies, or numbers, or facts. It was based on raw emotion.  And politics is an emotional game.  And it was the most powerful emotion there is, fear.

How do you measure emotion? How do you capture it in a database? What are its implications? How much impact will it have? How do you fold that into the poll and survey data? And in some cases you see it indirectly, like with the home made signs…and it is just not captured by any system. This was critical missing data. This was the stuff that tilted the results by just a few points here and there. This was the key to the entire thing. And it was not accounted for by almost anyone…except Trump’s folks. They could see it, sense it, feel it.

In the end, the ground game didn’t matter. The experience didn’t matter. The policies, the bad behavior, the offensive language, none of it mattered. And what mattered most was left out of everyone’s model. Maybe we’ve dismissed the “gut feel” a little too soon. Maybe we’ve become a bit too smug about our abilities to understand and know everything based on our data alone. Maybe we need to combine our data analysis with our hunches, our feeling, our gut. Maybe that slightly softer, less mathematical approach is the one that wins the day. I know it did on November 8th, 2016.

Arrange a Conversation