AI Return on Investment Audit
No baseline means no measurement, whatever the report says.
What is AI ROI audit?
An ROI audit independently checks whether a claimed return from an AI system actually happened, testing the baseline, including the full running cost, and separating the system contribution from everything else that changed.
Claimed returns are usually calculated by the team that built the system. That is not an accusation of dishonesty. Anybody measuring their own work chooses reasonable looking assumptions, and the reasonable looking ones happen to favour the result.
The first question is whether a baseline was recorded before deployment. If it was not, the return cannot be measured, and reconstructing one afterwards produces a flattering number because whoever builds it already knows what it needs to show.
The second is the denominator. Build cost gets quoted. Running cost, the staff time to operate it, maintenance and the hours somebody spends managing the supplier usually do not, and together they frequently exceed the build.
The returns we will not verify
Duara independently verifies claimed returns on AI systems for organisations in Kenya.
We do not verify returns on systems we built. Checking our own arithmetic and presenting it as independent would tell you nothing you could take to a board.
Who this is for
- A supplier is claiming a strong return
- You must justify last year AI spend
- You are deciding whether to expand or stop
- The numbers were produced by the team that built the system
Industries we serve
- Banking and financial services
- Manufacturing
- Insurance
- Retail groups
- Logistics
- Any organisation with AI investment to justify
Problems this solves
These are the reasons claimed returns survive without ever being tested.
- The baseline was never recorded. Without a measurement taken before deployment there is nothing to compare against, and the gap gets filled with an estimate made afterwards.
- Returns calculated by the builders. Anybody measuring their own work picks assumptions that look reasonable and happen to favour the outcome.
- Running cost left out. Build cost appears in the calculation while operation, maintenance and supplier management do not, though together they often exceed it.
- Everything else that changed ignored. Volume grew, staff changed and the season turned during the same period, and all of it gets credited to the system.
What we deliver
Independent verification of the baseline, the full cost of ownership included, the system contribution separated from other changes, and a plain statement where the numbers cannot support a conclusion.
Independent verification of the claimed return
Full running costs included, not build cost alone
A clear statement where the baseline is missing
Other changes in the period accounted for
A range rather than a false point estimate
A recommendation to expand, adjust or stop
Features
- Baseline tested first. Whether a measurement was taken before deployment decides what any later calculation can honestly claim.
- No reconstruction. We will not build a baseline after the fact, because a baseline built by somebody who knows the required answer produces it.
- Full cost of ownership. Operation, maintenance and supplier management included, since those are the costs that continue after the build stops.
- Attribution separated. What else changed in the same period is identified, because a business rarely holds still while one system is measured.
- Ranges where a figure would mislead. A supportable range is more useful to a board than a precise number the evidence cannot carry.
Benefits
- Board numbers you can defend. An independently checked figure survives questions that a supplier calculation does not.
- Expansion decided on evidence. Scaling a system that worked is sound, and scaling one nobody verified multiplies an unknown.
- Hidden costs surface. The running costs left out of the original case are usually the difference between a good return and a poor one.
- Measurement starts now. Where no baseline exists, establishing one from this point makes the next decision measurable even when the last one was not.
What a real return calculation includes
- Baseline evidence. What was measured before deployment, and whether it was recorded at the time rather than remembered.
- Build cost. What was spent to create it, which is the part everybody already has.
- Running cost. Licences, usage charges and infrastructure, which continue for as long as the system does.
- Operating time. The staff hours spent running, checking and correcting it, which rarely appear in any case.
- Supplier management. The internal hours spent managing the arrangement, a real cost that nobody bills for.
- Attribution analysis. What else changed in the same period, and how much of the movement it explains.
Attribution is the hardest part and the least discussed. Output rose during the period, and so did volumes, staffing and seasonal demand, so isolating what the system contributed usually produces a range rather than a figure.
How the process works, step by step
An ROI audit usually takes two to four weeks, and it is faster where a baseline exists than where we have to establish that one does not.
-
Step 1
Scoping
We agree which system and which claimed return is being examined, and what decision the answer has to serve.
-
Step 2
Baseline established or ruled out
Whether a pre deployment measurement exists, which determines what can honestly be concluded.
-
Step 3
Full costs assembled
Build, running, operating and supplier management together rather than the build alone.
-
Step 4
Attribution tested
What else changed in the period, and how much of the improvement it accounts for.
-
Step 5
Verdict delivered
Expand, adjust or stop, with a range where a single figure would overstate the certainty.
Quality assurance and how we work
The commitments attached to checking somebody else numbers.
- We do not reconstruct baselines. One built after the fact by somebody who knows the required answer will produce it, and we would rather report that none exists.
- We do not verify our own systems. Checking our own arithmetic and calling it independent gives a board nothing it can use.
- We include the costs nobody bills for. Internal hours spent operating and managing a system are real, and leaving them out is how a poor return looks acceptable.
- We give ranges when a figure would mislead. A supportable range is worth more than a precise number the evidence cannot carry.
- We will recommend stopping. Adjust and expand are not the only available answers, and an audit where they were is not worth commissioning.
Frequently asked questions about AI ROI audit
Can a return be measured without a baseline?
No. Without a measurement taken before deployment there is nothing to compare against, and every later calculation is an estimate wearing a number.
Why not reconstruct the baseline?
Because whoever reconstructs it already knows what the answer needs to be, and reconstructed baselines are reliably flattering as a result.
What should we do if no baseline exists?
Start one now. A return you can measure from this month forward is worth considerably more than a figure invented about last year.
Is a supplier calculated return dishonest?
Usually not. Anybody measuring their own work chooses assumptions that look reasonable and happen to favour the result.
What costs get left out most often?
Running cost, staff hours spent operating and correcting the system, and the internal time spent managing the supplier.
Do those costs really matter?
Frequently they exceed the build cost, which is why a return calculated on build alone can invert once the full picture is included.
What is attribution?
Separating what the system caused from everything else that moved. Volumes, staffing and seasonality change during the same period.
Why give a range instead of a figure?
Because a precise number the evidence cannot carry misleads a board more than an honest span does.
Would you verify a system you built?
No. Checking our own arithmetic and presenting it as independent gives a board nothing it can rely on.
Can the audit recommend stopping?
Yes, and that answer has to be available. An audit where the only outcomes are adjust and expand is not worth commissioning.
How is this different from a productivity audit?
That measures usage and task time. This tests whether the money spent produced a return, including the costs nobody counted.
What if the return is real but small?
That is a legitimate finding and often the most actionable one, since it usually points at running costs rather than at the system itself.
How long does an ROI audit take?
Two to four weeks, and it moves faster where a baseline exists than where we have to establish that none does.
Can it inform an expansion decision?
Yes, and that is the strongest reason to run one. Scaling a system nobody verified multiplies an unknown rather than a success.
Do we need the supplier to cooperate?
It helps and is not essential. Cost records and system data usually establish enough without their participation.
What if the numbers were already presented to our board?
Then verification matters more rather than less, and a correction found internally is far better than one found externally.
Does it cover systems still being built?
Not usefully. What it can do is make sure a baseline is recorded now, which is what makes later measurement possible.
Can you check several systems at once?
Yes, and comparing them frequently shows that spend and return are distributed very differently from what anybody assumed.
Who sees the findings?
Whoever you nominate. The report contains an assessment of decisions your organisation already took, so distribution is usually narrow.
What does an ROI audit cost?
The number of systems and the state of the cost records determine it, quoted after scoping.
Glossary of terms
Return measurement terms in plain language.
- Baseline
- A measurement taken before deployment, without which a later comparison has nothing to compare to.
- Total cost of ownership
- Build, running, operating and management costs together rather than the build alone.
- Attribution
- Separating what a system caused from everything else that changed in the same period.
- Reconstructed baseline
- An estimate of prior performance made afterwards, which reliably favours the person making it.
- Operating time
- Staff hours spent running and correcting a system, a real cost that appears in almost no business case.
- Supportable range
- The span the evidence genuinely allows, offered instead of a precise figure it cannot carry.
Related services in AI Audit Services
Talk to us about AI ROI audit
Find the business case for your AI system and check whether it includes running cost. Most stop at the build.
Start your project