Evaluating Tech: 10 Observations

Whether you’re evaluating a new CRM platform, a new marketing automation suite or deciding to implement a data management platform, one thing is for sure…technology partnerships can make or break a business.

The right solution could be the defining factor in delivering operational efficiency, commercial success and competitive advantage, whilst choosing the wrong solution could bring business operations crashing down around your ears. No wonder then that digital and technology consultancy now accounts for over 27% of the aggregate fees made by consultancy firms in the UK (UK Consulting Industry Statistics, 2016).

If the technology is so important, then the process of evaluating the technology must be equally important, right? But how many think about the process itself and how to get the best results? In the past several years I’ve been part of dozens of procurement processes as both a buyer and seller. I’ve seen the good, the bad and the downright ugly when it comes to tech evaluation processes. Below are some of my observations and tips for those looking to evaluate:

1. Understand where you are, so you can see where you are going 

It is vital to have a clear understanding of your existing infrastructure and technology stack, how it fits together, and how information and data are required to flow from one system to the next. Without this blueprint, you can’t hope to fully appreciate where this new technology platform will “fit in”, how it may overlap with existing partners, and where it will bring incremental functionality and value. In other words, before evaluating new technology, be sure to take an inventory of your current set-up and keep a blueprint for reference.

Tip: Share this blueprint with potential vendors, make sure they are aware of how you envisage the data and information flow and be sure to request information on their direct and indirect relationships/integrations with partners across your tech stack.

2. Know your requirements, embed them throughout the process

Often times, businesses begin evaluation processes without clearly identifying or prioritising their requirements. Understanding the overarching business objectives and translating these into functional requirements before launching the evaluation process is imperative to its success. Establishing requirements helps (1) to clearly focus the scope of the evaluation project, (2) to shortlist the type of vendor you should be evaluating (3) to identify and score each vendor’s capability to deliver on those specific requirements, (4) to remain on task during the process and have a continual source of reference, and (5) to help in constructing the implementation and deployment schedule once a vendor is selected.

Tip: Remember that requirements aren’t just for the Request for Information (RFI). We commonly see requirements established in the initial RFI that are then lost or forgotten through the process. Make sure to continue to refer back to your overarching business objectives and high priority requirements throughout the process.

3. Two-way information flow

In a successful evaluation process, information has to flow both ways. Vendors need to understand your business, your challenges, your infrastructure and your requirements in order to put their best, most relevant response forward. The more information you share, the more customised the vendor’s response and the more insightful and valuable the process will be.

Tip: Be open with requirements, priorities and what you think a good partnership looks like. Make sure you are open and honest with vendors and yourself about budgets and costings, don’t hope to negotiate a vendor down by 60% at the end of the process. It will only end in both parties being disappointed.

Get Expert Help and Advice for your Digital Transformation

4. Don’t just keep repeating the RFI

RFIs and RFPs are not interchangeable terms. They describe different information gathering exercises and should occur at different stages in the evaluation process:

(1) RFI – The Request for Information process should provide background information on the project and key requirements, and should inform relevant suppliers of your intention to complete an evaluation, collecting discreet information on the capabilities of vendors across a range of subjects, including:

– Technical Requirements

– Functional Requirements

– Service Requirements

– Privacy & Security Requirements

– Project Delivery Requirements

– Commercial & Contractual Requirements

(2) RFP – The Request for Proposal process should aim to solicit a proposal from a vendor based specifically on the delivery of pre-established requirements. It should outline the following:

– Proposed Solutions to Established Use Cases

– Implementation Schedule & Delivery Roadmap

– Client Service & Support Team Structure

– Case Studies & Client References

– Full Costing Framework

Tip: A simple way to delineate between the RFI and the RFP is to think of the RFI as providing information on the capabilities of a technology vendor, and the RFP providing information on how the technology vendor will leverage its capabilities on your behalf, within the context of your business, in solving your use cases.

5. What’s the Score?

The evaluation process will often be complex, lengthy and densely packed with information. It can be hard to keep track of the information flow and maintain a view on performance. Scoring processes are often overlooked and after running extensive RFI and RFP processes it can be hard to differentiate clearly between offerings. Maintaining a scoring matrix throughout the evaluation process will make the ultimate decision making process far easier.

Tip: An analytical process of review can also help to mitigate against the politics of cross-functional team decision making. Within any group, there will be different perspectives and preferences, and an analytical scoring matrix helps to reduce noise.

6. Shortlist with care

The process of creating a shortlist of 4-5 vendors based on the initial requirements is arguably both the most important, and simultaneously the most overlooked element to shortlisting as you go from a list of 5,000 tech vendors to just a handful, and often do this very quickly based on market perception, brand awareness, and generalised assumptions. After the initial shortlist, there is often a disqualification process after each distinct step in the process, with some vendors making it through to the next round and others being removed from the process at that point. As opposed to this being a “natural” process of elimination, we believe there should be a definitive rationale as to why vendors are being removed from the process, highlighting clear deficiencies where they came short of the requirement relative to their competitors. Keeping vendors in the process for the full length of the evaluation is not necessarily a sign of an inefficiency, however ultimately selecting the wrong partner because you “had to make a cut” after the RFI is.

Tip: Spend more time upfront on the initial shortlist by checking requirements against a range of potential vendors. Evaluate partners that may not necessarily have been in your original consideration set and don’t be offended if some vendors politely refuse to pitch. Ask them why they feel their capabilities may not match the requirements you have and use this as an information gathering exercise to inform the remainder of the process.

7. Test…but don’t procrastinate 

Live tests, sandboxes, and proof of concepts can be great ways to inform the ultimate decision-making process, but you shouldn’t embark on these activities simply for the sake of it. Testing requirements should be clearly established early on in the process, and vendors should be aware that you wish to initiate a test prior to engaging in the evaluation. The test framework should be clear, specific and scientific in design and execution, with key performance indicators and a clear vision of what success looks like.

Tip: Tests can extend the evaluation process significantly, and may not be free of charge. If you seek to complete tests using multiple vendors, ensure you have resource allocation and budget, and that timelines are adjusted accordingly. Deciding to run tests half-way through an evaluation can lead to major issues with timings.

8. Teamwork makes the dream work

People power is quite often the difference between success and failure in technology implementation and having a good working relationship with your suppliers is vital to the health and wellbeing of the partnership. Most technology vendors have a pre-contract and post-contract team structure, so make sure you meet the team that you will be working with after signing on the dotted line, not just the sales representatives. As part of the RFP process, you should expect to meet with your account management team or at least understand exactly how the support team is structured and how the relationship will be managed on an ongoing basis.

Tip: When meeting with the account management teams, highlight relevant experience, backgrounds, and expertise. Also try to assess personality clashes, which could become a problem later down the line.

9. In record time

Evaluation processes take time but should be condensed as much as possible to ensure information is retained, fresh and relevant. Don’t plan to complete evaluation processes if you have planned absences within the team, holidays etc. and try to keep spacing’s between each individual step in the process as short as possible, whilst providing enough time for vendors to properly formulate responses. It’s a balancing act, you don’t want to space out sessions by too great an extent but you need to make sure enough time is provided for vendors to deliver high-quality work.

Tip: Map out the full evaluation process before beginning and bear in mind a few oft-forgotten items (1) the time it will take for you to produce analysis on each stage, (2) travel schedules, and (3) office space and meeting rooms (yes having an appropriate space for an appropriate amount of time is important).

10. One more for the road

Feedback is incredibly important to both the winning and losing parties in an evaluation process. Some see providing feedback as benevolence, others as a duty of care. We tend to agree with the latter. Most vendors spend a huge amount of time, energy and resource in completing evaluation processes and it can be bitterly disappointing to hear you have lost an opportunity, but feedback is incredibly valuable in understanding and making sense of why you lost, providing some closure and helping you to improve for the future.

Tip: make sure feedback is embedded into the evaluation process, that you have a feedback template and time set aside to present feedback to each individual vendor.


Arrange a Conversation 


Article by channel:

Read more articles tagged: Featured