The secret to managing risks well in agile environments

As a general rule, technology systems don’t like being fiddled with. Changes have a tendency to introduce unwanted…let’s call them ‘features’. Security vulnerabilities, hidden calculations and unexpected system downtime are all common byproducts of change.

Risk management in technology change is really important.

When IT still needed to be spelt out as Information Technology, intelligent people with an understanding of risk developed an idea of approaching systems development nice and carefully. Thinking in a clear, linear way, they created an approach that provided comfort throughout that the time investment they were about to make was justified. The key control steps were easy to understand:

• Detailed business case before wasting time on something not justified
• System design for the full solution before wasting time in development that wouldn’t finish
• Full technical testing before wasting users’ time in a system that didn’t work
• Full user testing before risking the change in the live environment

Laid out in the form of a Gantt chart, it looks a bit like a waterfall (to someone who’s never seen a waterfall before).

But a waterfall approach doesn’t tend to deliver the best user experience.

As technology became more widely available, a good user experience started to be a defining factor in choosing systems. That’s not waterfall’s strong suit. Systems developed in waterfall tended to be overly functional, and so a need for software development approaches that valued the quality of the user experience alongside its functional requirements was birthed.

Several variations on a theme evolved in the later part of the 20th Century, such as Evo, RAD and Scrum, and these were gathered into one, united, flexible approach to software development in 2001 in the form of The Agile Manifesto.

The problem is that the Manifesto, at its core, removed the controls waterfall had introduced.

The Manifesto is based on four values:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

For an industry that had spent a long time getting comfort over risk management thanks mainly to well-defined processes, mandated tools, comprehensive documentation, thorough contract negotiation and clearly communicated plans, this is a step too far.

Isn’t it a shame that such a great idea misses the mark so much?

But surely the results speak for themselves. In a lot of waterfall-based projects, a lot of time is spent upfront, defining business requirements, detailing technical specifications and engaging with suppliers who can fulfil those needs. Then the developers get to work and often for many months aren’t heard from, unless they’ve discovered issues that mean they need more time (and they frequently do).

Then the project hits testing.

Issues start to appear all over the place. An interface doesn’t do what it ought to. A calculation keeps rounding halfway through. Two of the data entry fields have been marked up the wrong way. Something’s missing altogether. A user remembers something that should have been raised ages ago. And so on.

All of the comfort that the upfront planning gave was false assurance, and so IT projects, particularly big ones, have consistently come in late and over budget. I’ve seen one particular article, published in 2011, quoted several times. The average project overspend in their review was 27%, but “one in six of the projects we studied [had] a cost overrun of 200%, on average, and a schedule overrun of almost 70%.”

These terrifying results are essentially impossible in a truly agile environment.

With a focus on customer collaboration, working software and responding to change – most typically daily meetings between developers and users, with fortnightly software releases – the worst that can possibly happen is that two weeks’ work needs to be revisited, and the system that ends up being delivered doesn’t include all the bells and whistles.

It turns out that those new values, when genuinely lived out, act as better risk mitigates than the carefully designed controls that form waterfall’s bedrock. There’s nothing wrong with them, but the world doesn’t work in straight lines.

Agile isn’t the solution to everything.

There are certain things that waterfall does very well, and agile doesn’t.

It forces individuals to sign their names against things, increasing individual accountability (there are downsides to this too – I wrote briefly about that culture change over here).

It tends to force certain key people to get involved in every change, such as information security and compliance, and you don’t want to miss something in one of those arenas.

And it’s such a mindset shift for some people that a hybrid approach is born, which seems to take the worst of both worlds (lots of time spent in meetings, accompanied by slow progress and late identification of issues).

All in all, when it’s done well, agile offers unique value. In many cases it can be better at managing risk than waterfall approaches, and that requires an intelligent, mature approach and a willingness to concede that there are perhaps better ways of working.

Arrange a Conversation