Thian-Peng Ter - An Architecture Valuation Framework for Software Investment

An Architecture Valuation Framework for Software Investment

February 4, 2019

Some of the buy-versus-build discussions can be difficult because making the wrong choice will cost more than expected. People in the debate either make an emotional decision based on what "feels the best" or make a rational decision but then it takes a significant amount of time to collect data. This type of discussion is intimidating to most people, and we prefer not to talk about it. In this article, I will describe a lightweight architecture valuation framework that you can use to have a productive conversation with others.

Defining Evaluation Criteria

Think about the last time you decide whether you should dine out or make a meal at home. You probably have a long day at work, so you want to treat yourself an evening out but with an added cost. On the other hand, cooking will save you some cash, but it can be tiring and time-consuming. In this case, your energy level, cost, and the need to treat yourself are your evaluation criteria, whereas dining out and making a meal at home are your options. After weighing these options against these criteria, you then decide on your dinner.

When it comes to buying or building software, like in the example above, you start with defining your evaluation criteria. Rather than doing it alone, it is best to collaborate with others in a group setting to build rapport and develop shared understanding, ideally using the Communities of Practice (CoP) approach. After gathering all your subject-matter experts, for example, your group could start with the following criteria and define what each means to your organization:

Cost

Implementation Cost Efficiency
Operational Cost Efficiency
Savings on Future Solutions

Business Agility

Speed to Benefit
Enabled Speed of Solutions

Architectural Quality

Performance
Availability
Flexibility
Security

Once you have identified and explained all the essential criteria, you will rate them for priority and weigh the different options against them.

Ratings

1. Set Priority Level for Your Evaluation Criteria

Before prioritizing your criteria, provide people with the scale and explain what each rating mean.

2. Score Your Options

You then assess how each option satisfies your criteria.

Weighing Your Options

Now that you have a set of criteria, you then assess how each option satisfies them. For example, the radar chart below shows how buying a particular piece of software favours both Cost and Business Agility over Architectural Quality.

Example of Comparing Alternatives

Performing Gap Analysis

With the evaluation criteria and options defined, you are now ready to conduct a gap analysis, as shown below.

Gap analysis does not need to be complicated because your primary goal is to have an objective discussion with others. So don't get too hung up on the details.

Define the gap of each option to be the difference between the criterion priority and the rating of that option.
Aggregate these gaps by adding their values together.
Determine the option with the smallest total gap, e.g. the Build option is your recommended choice.

It is a common pitfall to only compare the different options without performing a gap analysis.

If you have not gone through the gap analysis, you would have chosen the Buy option. After weighing the options against your criteria, your recommended choice is the Build option.

Making Framework Available to Others

Buy-versus-build discussions are frequent as you continue evolving the architecture to meet new business needs. It is helpful to first define the evaluation criteria with a small number of options and share them with others. You can then leave the priority of the evaluation criteria to the individual or the team to decide. Once they have agreed on the priority, they can perform the gap analysis and recommend the option.

If you are interested in trying out this framework, feel free to make a copy of this Google Sheet.