Why Do Analytical Pilots Fail?

It is said that some astounding proportion of BI and analytics efforts fail. Depending on the context, that number appears to range from 50% to 80%. Certainly, numerous debriefs have been conducted on what worked and what did not work; many have opined on the top reasons for failure. So, why does this continue to happen?

Take analytical pilots. (Here we refer to pilots whose main concern involves analyzing the data and not simply implementing a tool–the latter deserves a separate discussion.)

Pilots are particularly important, because the resulting decisions shape the course of what to come. At the risk of stating the obvious, organizations conduct them to do something with little to no precedent, and pilots are a financially prudent way to see if it works before investing in a larger-scale capability; starting small does provide an opportunity to work out the kinks, while also allowing organizations to plan better.

However, the fundamental reasons for the start-small approach deserve more careful thought in analytics. Specifically, we should ask whether the organization is ready for the consequences of the analysis results, and recognize that the transformation expected from the positive conclusion does not happen naturally. A pilot intended to prove the value of analytics is especially tricky, as the very need to prove may indicate that someone in the organization has not yet bought into the idea of the consequences–applying analysis results to make changes, and changes are uncomfortable. There is merit to convincing the unconvinced, but the degree to which the entire organization becomes convinced is a huge factor in whether there is a realistic future for analytics beyond the pilot.

That is, the organization must have the collective desire to be data-driven and have the next steps already defined, ready to accept change. The main goal of the analysis should be simply to prove the sufficiency of the business impact, with everything else already in place or ready to be executed immediately upon the completion of the analysis. Unfortunately, many non-financial planning and decisions are put on hold until the results of the analysis are available; the situation is exasperated with the unconvinced or the marginally convinced. We have seen pilot analysis executed, only to be followed by lack of priority, a long time to define the next steps, and finally the demise.

But a data-driven culture does not deprioritize the conversion of data-driven efforts into business results. First, if for some reason a well-selected and well-executed analytical pilot end up with less-than-favorable results, it has others in the wing waiting to benefit from the learning. Second and more importantly, for a pilot to be effective, those who will consume its results must be willing, able, and empowered to do so immediately–accept and action on those results to change themselves. The real challenge with analytics is that, without the resulting operational or strategic change–i.e., non-analytical change–it has no business value. And pilots cannot succeed with no business value. A data-driven culture is all about establishing an ecosystem of consumption of analytics throughout the organization and less about acquiring tools and data scientists. Having experience and capabilities in analytics is not a prerequisite.

I am not ruling out the possibility that there exist organizations for whom analytics makes no sense whatsoever, but I have yet to come across one in my nearly two decades of looking at analytics and analytical practices. I have, however, seen plenty that were not ready to consume. I am willing to bet that some analytics is sufficiently positively impactful to well more than the 20-50% of the opportunities as suggested by the failure rate. I am also willing to bet that a substantial portion, if not the majority, of the failures never came close to implementing the non-analytical changes needed to understand the business value.
 
Are you going to be content with continuing the trend of failure, or are you going to challenge your business to transform?