Analytics, Technology Development Lifecycle, and Family Vacations

One of the prevailing challenges in making analytics successful is obtaining quick wins. What the businesses often need is an analytics equivalent of agile sprints approach, in which actionable results are delivered in short cycles; yet for many, this continues to escape reality. Too often an elaborate analytical project is planned or carried out, stemming from insistence (on rigor and perfection), naiveté (about all that are possible), or neglect (of the original intent).

Contrary to the common perception, an effective execution of “agile analytics” is almost all about design rather than about tools and/or technology; the speed to insight is driven almost entirely by how the analysis itself is designed. While tools and technology can sometimes help, they cannot make up for poor design–they will simply deliver an invalid solution a tad faster. And when the analysis is sufficiently simple, a wrong design delivers with less effort a solution to the wrong problem. This is like the worst case of “are we there yet?”–wrong directions and/or wrong destination; having a faster car does not correct for either, and all the passengers are hungry and cranky.

Many activities related to analytics have direct parallels in technology, although this often eludes even the highly skilled analytics professionals. The basic technology development lifecycle framework–Discover, Design, Develop, Deploy, or any of the other widely accepted variants thereof–applies to analytics as well.  However for analytics, most of the attention is currently going to the “Develop”–that is, where the analytical techniques are applied–that the other three phases are practically being ignored. This has an impact not only on the feasibility of agile analytics but on the effectiveness of analytics overall.

Just like in technology, the “Develop” in analytics–effective application of analytical techniques to obtain insights or build an algorithm–must be based on good functional requirements that are designed to build the business needs as well as operational constraints into the solution. Then, the analyst’s job is to use the best techniques to fulfill those requirements through development. The challenge is, data scientists are often expected to do both the “Design” and the “Develop” based on their ability to apply analytical techniques–that is, based solely on their ability to do the “Develop.” This is equivalent of expecting technology developers to be great architects based on their ability to code. The technology field is now mature enough to realize this, but somehow the analytics community has yet to acknowledge the same. To further complicate the matter, many data scientists implicitly believe they can design based on their ability to develop, since the distinction is not very well understood even in the advanced analytics community.

What makes an analytical professional great at analytical design? That is a discussion for another time. However, it suffices to say that there are far more expert analytical developers than there are professionals who excel in analytical design. And as with anything else, it pays off to get it right up front–getting the solution design right is critical in setting the right expectations. Then, if the ecosystem of analytics consumption is in place (see earlier post), you will have two major pieces of foundation needed toward getting the real value out of analytics that are missing from so many of the analytical endeavors today.

Image courtesy of digitalart at FreeDigitalPhotos.net