An Architecture Valuation Framework for Software Investment
February 4, 2019
Some of the buy-versus-build discussions can be difficult because making the wrong choice will cost more than expected. People in the debate either make an emotional decision based on what "feels the best" or make a rational decision but then it takes a significant amount of time to collect data. This type of discussion is intimidating to most people, and we prefer not to talk about it. In this article, I will describe a lightweight architecture valuation framework that you can use to have a productive conversation with others.
Defining Evaluation Criteria
Think about the last time you decide whether you should dine out or make a meal at home. You probably have a long day at work, so you want to treat yourself an evening out but with an added cost. On the other hand, cooking will save you some cash, but it can be tiring and time-consuming. In this case, your energy level, cost, and the need to treat yourself are your evaluation criteria, whereas dining out and making a meal at home are your options. After weighing these options against these criteria, you then decide on your dinner.
When it comes to buying or building software, like in the example above, you start with defining your evaluation criteria. Rather than doing it alone, it is best to collaborate with others in a group setting to build rapport and develop shared understanding, ideally using the Communities of Practice (CoP) approach. After gathering all your subject-matter experts, for example, your group could start with the following criteria and define what each means to your organization:
Cost
- Implementation Cost Efficiency
- Operational Cost Efficiency
- Savings on Future Solutions
Business Agility
- Speed to Benefit
- Enabled Speed of Solutions
Architectural Quality
- Performance
- Availability
- Flexibility
- Security
Once you have identified and explained all the essential criteria, you will rate them for priority and weigh the different options against them.
Ratings
1. Set Priority Level for Your Evaluation Criteria
Before prioritizing your criteria, provide people with the scale and explain what each rating mean.
2. Score Your Options
You then assess how each option satisfies your criteria.
Weighing Your Options
Now that you have a set of criteria, you then assess how each option satisfies them. For example, the radar chart below shows how buying a particular piece of software favours both Cost and Business Agility over Architectural Quality.
Performing Gap Analysis
With the evaluation criteria and options defined, you are now ready to conduct a gap analysis, as shown below.
Gap analysis does not need to be complicated because your primary goal is to have an objective discussion with others. So don't get too hung up on the details.
- Define the gap of each option to be the difference between the criterion priority and the rating of that option.
- Aggregate these gaps by adding their values together.
- Determine the option with the smallest total gap, e.g. the Build option is your recommended choice.
It is a common pitfall to only compare the different options without performing a gap analysis.
If you have not gone through the gap analysis, you would have chosen the Buy option. After weighing the options against your criteria, your recommended choice is the Build option.
Making Framework Available to Others
Buy-versus-build discussions are frequent as you continue evolving the architecture to meet new business needs. It is helpful to first define the evaluation criteria with a small number of options and share them with others. You can then leave the priority of the evaluation criteria to the individual or the team to decide. Once they have agreed on the priority, they can perform the gap analysis and recommend the option.
If you are interested in trying out this framework, feel free to make a copy of this Google Sheet.