Measuring return on investment (ROI) and proving value on technology and digital projects has plagued business and tech people for decades. Organisations often feel the need to jump on a trend and invest heavily – but this is usually before they know what they’re trying to achieve, how the technology will benefit them and how success will be measured. AI is no different.
As AI adoption accelerates, the pressure to demonstrate clear and tangible returns is becoming more pressing – but measuring ROI for AI is not very straightforward. Traditionally, ROI tends to focus on cost savings or productivity gains – but this perspective is a bit too narrow to capture the full value of AI.
Quantitative outcomes of AI, such as revenue growth, productivity improvements and risk reduction are important, but they rarely tell the full story. AI also delivers qualitative benefits, such as improved decision making, greater consistency, enhanced customer experience and increased business resilience. All fairly intrinsic variables, which are difficult to measure.
We think that to measure ROI accurately and to steer AI initiatives effectively, organisations need a systematic approach made up of five core principles:
1. Measure the costs
The first step in measuring ROI starts with understanding the full cost of AI development. AI investments tend to fall into two categories: foundational and use-case driven. The foundational stage consists of frameworks, internal regulations and software licenses that underpin all AI initiatives. These are typically one-time investments that need a lot of upfront time and capital but enable faster deployment of future use cases. Use-case-driven investments are more variable and scale with the number of applications in production. Each use case incurs development costs for engineers and subject matter experts, plus the specific IT infrastructure that’s required for the solution. Once it’s deployed, ongoing costs usually prevail for cloud computing, maintenance, and continuous improvement. Change management programmes that train employees how to use the new way of working usually need additional investment.
2. Define success criteria per use case
Ultimately, ROI is highly dependent on the use case – measuring ROI is only as effective as the use case is robust. Whether it’s a fraud detection model, an asset monitoring system, or an AI-driven customer service assistant, each use case has different objectives, so it needs different success criteria. The key question is not ‘what is the ROI of AI?’ but ‘what does success look like for this specific use case?’ This requires clarity upfront on the parameters that matter and the organisational changes required to capture the value of AI. You need to know which processes will be affected, which decisions are expected to improve and where the impact should become visible over time. If organisations can’t answer these questions, they will find it difficult to define actual returns.
3. ROI benchmark – to prioritise use cases
In theory, to maximise AI returns, organisations have to prioritise initiatives that offer the highest expected ROI. The challenge is that different AI applications deliver value across different dimensions that are difficult to compare. A customer service assistant may reduce the need for manual labour and increase customer satisfaction, while a fraud detection system primarily reduces risk and improves regulatory compliance. Indirect returns, such as customer satisfaction and compliance improvements, are harder to quantify but obviously still justify major investments.
Objective comparison requires a comprehensive framework, consisting of indicators that quantify direct and indirect returns. Once these parameters are defined, an ROI benchmark can be established before implementation. This benchmark provides a reference point against which outcomes can be assessed after deployment and should reflect both the ambition of the use case and the regulatory context in which it operates.
4. Be nimble – change plans when needed, or exit early
Guaranteed, behind every business success is a list of failed investments. When initiatives do not meet the expected returns, it is important to understand why. Sometimes, the approach can be changed to transform the initiative into a success. For example, we developed a chatbot for a large industrial player to support their engineers. Despite the very high accuracy of the answers, the response time of two minutes was too high for practical use. By changing the solution and an agreement to trade elements of quality in the response for the rapid delivery of answers, the chatbot is still a success. Sometimes, the hurdles to making an initiative profitable can’t be overcome. In those cases, it is important to stop early to avoid further losses while making changes and applying lessons learned on use cases that are in progress.
5. Monitor returns over time
Returns are not measured by a snapshot. Organisations must recognise that AI systems evolve over time and often deliver increasing returns as adoption and maturity grow. For example, we developed a computer vision solution for a railway company to monitor damaged sleepers. Because of the diversity of damages and lack of available examples, the first prototype did not have sufficient accuracy and still required a lot of human inspection. Through iterative improvement, the accuracy of the solution gradually improved. The eventual solution outperformed human inspectors and could inspect thousands of kilometres of track within days. This increased railway safety and reduced human inspection time by 17 FTE. This wasn’t realised immediately, but over time.
By applying these five different principles, ROI becomes more than a retrospective justification of investment. It becomes a steering mechanism that supports the prioritisation of use cases, the design of AI models and decisions around scaling. Organisations that embed ROI thinking into their AI initiatives are better positioned to move beyond experimentation and translate AI adoption into sustainable business value.
If you would like to speak to Valcon about how to implement a framework to measure ROI in your AI implementation, please get in touch with: [email protected] or [email protected]
