The Big Switch – DevOps in financial services

DevOps is becoming the delivery style du jour as financial services firms face the ongoing challenge of responding to changing customer demands, coping with regulatory change while remaining competitive.

According to a global study of 1300 senior IT practitioners surveyed by Freeform Dynamics for CA Technologies, businesses fully embracing DevOps and Agile practices are achieving a 60% increase in revenue and profit growth. These organizations are also 2.4 times more likely than their mainstream peers to see business expansion at a rate of over 20%, the study adds.

In DevOps, operations and development engineers work together in the entire service lifecycle, from design through the development process to production operations, support, and maintenance. Under the approach, operations staff use many of the same Agile techniques as developers for systems work, such as test-driven development and continuous integration.

Benefits for financial services firms

The main upside of DevOps for financial services firms is error-proofed deployment to production environments, which results in fewer failures and unwanted rollbacks, according to Simon Baker, chief executive at Energized Work, an Agile consulting firm serving large global banks and investment companies.

This, says Baker, is due to two things: the elimination of manual intervention by operations staff added to the replication of production infrastructure and configuration – as much as can be afforded – on near-production environments such as pre-production or staging, user acceptance testing or performance testing. According to the expert, when all environments are the same there are no failures due to configuration differences. He see other benefits that include:

Shorter end-to-end lifecycle by removing manual wait-times for scheduled deployments to lower environments such as development, demo, and testing, and earlier identification of mismatches between what a system is capable of, given its design, and what the production environment requires. The production environment is not the place to find out the system or the infrastructure is not fit-for-purpose.

Well-executed adoption of DevOps practices shortens in-process lead times while improving quality, which in turn enables continuous integration, regular deployment, and continuous delivery. According to Baker, this means being better equipped to ward off the threat of disruption from challenger banks and other fintech startups, as well as those of traditional financial organizations who are starting to get this right. He adds:

Until now, established banks have not had the threat to make them worry about losing customers due to a lack of new features and services or better user experiences because their offerings were largely the same. Infrastructure has not typically been an area for innovation but DevOps practices, SaaS tooling and cloud technology present the opportunity to do more with infrastructure, from A/B or multi-variate testing that drives increased customer traction, to enhanced plugability of systems that enables continuous modernization.

The security threat – and opportunity

Agile, DevOps and continuous delivery approaches are often viewed as threatening when misunderstood, hence why many financial firms have incubators that allow experimentation with these techniques outside their normal restrictions and at a lower risk. The challenge then becomes the integration of successful experiments into the core business. Here, says Baker, security can be a major inhibitor:

Changes to well-practiced operations in a financial organization require diligence and care given the stakes are so high. However, there are ways for security practices to coexist with responsible DevOps. DevSecOps is becoming an increasingly recognised concept among the DevOps community with tools and solid practices emerging.

Financial organisations tend to separate development and operations teams, which fosters distrust and inhibits the flow of deployments, says Baker. This issue is compounded by the need for accountability and an assumption that all security flaws can be discovered if enough checks are put in place:

While the discovery of security vulnerability tools like Heartbleed, Meltdown and Spectre help, they do not reduce the importance of automated real-time monitoring and alerting, intrusion detection, and the critical ability to respond quickly to incidents, by, for example taking certain services offline. A DevOps team that is separate to development teams often provides just ‘better ops’ rather than practice DevOps.

On the other hand, there is a lot of interest from financial services firms in solving security and compliance problems in ways compatible with DevOps. An example is insurance and asset management giant Allianz, where DevSecOps coupled with cloud and Agile development ushered in a step change in IT delivery. Commenting on the benefits of the methods so far, Allianz chief information officer, Jacob Abboud, says:

[DevSecOps] brought about tremendous benefits to both the IT organization and the business bottom line. We have taken a 24-month programme of work to develop new customer journeys for Quote & Buy and condensed them for a successful delivery in seven months accelerating time to market.

Further evidence of tangential benefits of the approach to Allianz is an increase of more than 50% in conversion rates since the deployment compared to the old capability. Abboud adds:

This has also had the effect of increasing the business confidence in our ability to deliver new capabilities quickly and cost-effectively, which propelled them to start a new wave of investment in the next phase of digital services that they would like to offer our customers.

Overcoming information security issues was a steep learning curve, says Abboud, but the approach enabled the team to fix security issues highlighted by the penetration testing in real-time, which was “very effective.”

Transforming the function

The adoption of DevOps at Allianz is also changing on the skills capability of the organization for the better and the insurer has been developing and maturing an internal capability with key skills to support this way of delivery. According to Abboud, the approach has introduced a “tremendous journey” of learning and adopting new technologies and ways of working. He says:

I have not seen so much commitment and enthusiasm from the team who, irrespective of where they sat organizationally, worked together with a common goal and new ways of working to deliver an important capability to our customers.

According to Abboud, the measures of success of the introduction of DevSecOps at Allianz included the transformation of the function into an empowered, multi-disciplinary team which is co-located. The approach of infrastructure as code is supported by tools and automation and is underpinned by continuous development and integration. In addition, change is fast and simple and team communication is instant.

When introducing DevSecOps, Abboud’s expectation was that this mode of delivery would become business as usual and the new benchmark for all software delivery. In terms of how Allianz is progressing towards that goal, the executive says that the firm is looking into its change portfolio in order to identify opportunities to explore the use of DevSecOps on “a multitude of projects and programmes with much success.” Abboud says:

We have been exploring the interoperability between the use of public cloud on AWS and our internal private cloud. In order to exploit our offshore applications development and management capability we have started a pilot to test a distributed approach based on the type of project and with the intention of a multi-phase rollout – it is all about the maturity of the organisation.

Overcoming challenges

When it comes to challenges related to DevOps, in some instances Allianz needed to modify its approach where the use of third-party software does not support the use in the cloud due to technical or licensing constraints. But this was not an impediment to the overall strategy. The CIO says:

We are on a journey and I am still driving towards this method of delivery to become the preferred method of delivery for all software development projects.

However, there are some difficulties that are more complex to tackle, such as skills availability, despite all the gains the company has had so far in that front. According to Abboud, Allianz has had to rely on a partner to provide the expertise. In addition, the fact that the firm’s starting point was based on team co-location did not make things easier. The CIO says:

This meant that our offshore team has had to be here in the UK for an extended period of time, which is not ideal, but nevertheless useful when we now start to consider progressing our implementation towards a distributed approach.

Beyond the cases where even the trailblazers suffer from problems such as lack of skilled professionals, there are other barriers to DevOps adoption among financial services firms, from inability to change organizational culture and lack of support from management to attempting to fit an agile style into a non-agile framework, says David Schumacher, a managing director at Accenture’s banking practice. He says:

In reality, DevOps change is not very different from other change projects, but it is critical for success. Sustainable DevOps introduction requires a holistic change management roadmap and C-level commitment to ensure long-term success.

According to Schumacher, steps required to overcome these challenges begin with understanding the status quo and change impact of the approach, then include creating a vision of the future organization and identifying and gaining commitment from stakeholders, creating an open communication plan, providing the proper training where needed and ensuring constant engagement with the organization.

Echoing Abboud’s experience with his team at Allianz, the consultant adds:

Overcoming the challenges brought on by DevOps will allow the organization to enable Agile delivery and increase software quality and technical enablement, as well as better cross-team collaboration and therefore organizational transformation.

My take

This is a classic case of ‘resistance is futile.’ The adoption of DevOps across financial services IT shops should not surprise given the many existential threats facing the sector.

Sector practitioners operating in highly regulated businesses agree that the DevOps approach will become widely adopted in the near future, especially where propositions are based on ecosystems of services provided by various providers through standard APIs.

I anticipate that Open Banking will act as a DevOps adoption incentive. As many incumbents have had trouble meeting the PSD2 deadline in January and will continue to struggle for many months to come, considering the modernization of software delivery is taking on an imperative of its own.

Image credit – via VMWorld and public images