Summary. Boards are being handed AI return figures that nobody had to verify. We build the systems those figures are about, and the returns we can evidence are not the ones that get reported. This is an account of what is measurable, what is merely reportable, and the decision instrument we use to tell them apart.
#The question that has no owner
The question a board asks is "what ROI are we getting from AI?", as though the number existed somewhere in the building and someone had simply not written it down yet.
Nobody has it. We are usually the ones asked, and what we can hand over honestly is narrower and stranger than the question expects: a stage that used to breach its timeout and now does not, a bill that fell while volume rose, a provider that could be swapped in an afternoon. None of those are what the question was reaching for.
PwC's AI performance study puts real numbers against the gap. It surveyed 1,217 senior executives, director level and above, 91% at publicly listed companies and 76% at companies with a billion dollars or more in revenue, across 25 sectors, with 85 respondents in Africa. The top 20% capture 74% of AI-driven returns. The most AI-fit companies show 7.2 times the AI-driven performance of everyone else. Africa sits at the global median on the fitness index while trailing the leaders on every dimension of performance, which the report reads, reasonably, as an execution problem rather than an adoption one.
The survey is competently run and the recommendations are sound. The difficulty is in the methodology note, where AI-driven performance is defined as the revenue and cost gains executives attribute to AI. Sixty management practices were then tested against that figure. The outcome and the practices come from the same person in the same sitting.
In the programme we built, a participant cannot claim an outcome. An outside party confirms it, or it does not count. Almost every published AI ROI figure is reported on precisely the terms that rule exists to reject.
#What is actually being measured
There is one figure in the study that does the argument's work for it. Among CEOs who had invested in AI, 23% reported revenue increases over the past year and 25% reported cost reductions. Read that plainly: on the most generous instrument available, an executive describing their own results, roughly three quarters of the people who spent the money could not point to revenue.
Both readings of that number are uncomfortable. If it is accurate, most AI investment is not returning revenue yet. If it is generous, which self-report usually is, the true figure is lower. There is no version where it supports urgency about falling behind, and it sits in the same document as the recommendation to invest more decisively.
The concentration figure deserves the same scepticism, from the other direction. The top 20% capturing 74% of returns is presented as a finding about AI. It is close to what you would expect from almost any class of business investment: venture portfolios, new product launches, store openings, acquisitions. A Pareto split is the normal shape of returns under uncertainty. It tells you that outcomes are dispersed, which is worth knowing, and it does not by itself tell you the top quintile did something reproducible.
| What gets reported | What was measured | The reflex | What would have to be true |
|---|---|---|---|
| "AI increased our revenue" | An executive's attribution, after the fact | Benchmark against the leaders | A pre-period, a comparable untreated case, and someone outside the team confirming the revenue |
| "AI-driven cost savings" | The distance between two estimates | Put it in the ROI model | A bill that fell on unchanged or rising volume |
| An AI fitness score | Self-described practice, regressed on self-described gain | Raise the score | An outcome measured by anyone other than the person describing the practice |
| "We have scaled AI enterprise-wide" | A count of deployments | Fund more pilots | One use case somebody outside the team would notice if it were switched off for a week |
Read the last column downward and the shape is clear. Every AI number that survives contact with an auditor has the same three parts: a baseline captured before the change, a population that did not get the change, and a party with no stake in the answer. Confidence in AI correlates with how well a company is run, and well-run companies report more of everything. A survey cannot separate the two, which is not a flaw in the survey. It is a limit on what a survey is.
#The return that looks like a retreat
The cheapest inference is the call you never make. In our multi-tenant publishing platform, candidate topics are gathered from public signals and scored deterministically before any model is invoked. The expensive stage only ever sees material already judged worth paying for.
That single decision moves more money than any model upgrade we have made. It also arrives on the board's desk as a falling AI bill, which reads as reduced AI investment, which reads as a lower fitness score.
Here is the cost asymmetry. A model upgrade that improves output by a few per cent is legible upward: it has a vendor, a launch date and a name. An architectural filter that removes most of the calls has none of those, and its evidence is a line going down. The framework rewards the first and is structurally blind to the second, so the organisations best at controlling AI cost are the ones most likely to be told they are underinvesting.
#Latency is a budget, and a budget is the only auditable AI number you own
A ceiling turns an improvement into a fact. Extraction in our financial intelligence platform ran per article, sequentially, and exceeded the pipeline's timeout. Restructuring the batch to run concurrently cut extraction time by roughly two thirds and brought the stage back inside budget.
Compare the evidentiary status of that with "AI helped us grow revenue". The extraction figure has a fixed corpus, a before, an after, a threshold that was breached and then was not, and a re-run anyone can repeat. It is falsifiable. Somebody can take the same inputs and show we were wrong.
The relocation worth making is this: performance work gets filed under engineering, and revenue attribution gets filed under strategy, when the evidentiary quality runs exactly the other way. The engineering number is the one that would survive a hostile reading. The strategy number would not survive a curious one.
This is why we set a latency ceiling before choosing an architecture rather than after. A ceiling is not only a service commitment. It is the instrument that converts a change into a measurable claim, because a threshold gives you a pass and a fail where a percentage only gives you an argument.
#The price of a model is what it costs to leave
Schema is the contract. When we moved entity extraction and sentiment off locally hosted transformer models and onto a hosted LLM, the interfaces and data structures stayed identical, and every downstream consumer kept working through the migration. Multiple providers now sit behind one interface, selected by configuration. A provider that degrades or reprices is a config change; a provider that disappears degrades the feature instead of taking the service down.
# the paid stage is the only stage that knows a provider exists,
# and it is not allowed to change the shape of what it returns
candidates = gather(sources) # free
scored = [c for c in candidates if worth_it(c)] # deterministic, free
results = extract(scored, provider=CONFIGURED) # metered
assert shape_of(results) == EXTRACTION_CONTRACT # true for every provider
What the shape buys is two things, and the second is the point. It makes the cost of the metered stage a function of a threshold we control rather than a vendor's price list. And it makes the exit cost knowable, because nothing downstream knows which path produced the data.
Now add the term the study does not carry. Metered inference is denominated in dollars. Revenue across the markets we build for is not. A use case that clears its hurdle rate this quarter can go underwater on a devaluation with no change in usage, no change in performance and no decision made by anyone in the company. Priced properly, an AI use case in this region is a position with currency exposure, not a project with a payback period, and "self-host where the volume justifies it" is a hedge before it is a cost decision.
#The two recommendations that are right and not yet actionable
Two of the study's seven cannot be answered with anything we have built, and both deserve better than silence.
Convergence is a rails problem before it is an ambition problem. The report identifies the largest growth pools as sitting between sectors, and notes that organisations in the region are the least likely to compete there. Both are almost certainly true. But cross-sector AI value assumes things that are infrastructure rather than strategy: data portability between institutions, an identity that travels across them, and the legal right to combine a customer's records from two industries. Where open banking has actually shipped and a consent regime permits the combination, "design for ecosystems" is a decision a board can take. Where it has not, the same sentence describes a destination with no road to it, and a CEO who accepts the advice will fund a partnership team to discover that the blocker was a data-sharing standard nobody has published. The recommendation is sound and the sequencing is missing.
Workforce trust is a governance artifact, not a training outcome. The finding is that workforce openness to AI in the region runs ahead of global peers while trust in AI-driven decisions runs behind, and the recommendation is skills, role redesign and decision support. Skills will not close that gap, because the gap is not about capability. People extend trust to a system whose limits they can see. In our own programme software the model classifies, personalises and flags, and it is prohibited from overriding a rule, granting an exception or determining who completes the programme. Those prohibitions are written down and enforced in code, and they are the reason a participant can be told what the system did and did not decide about them.
That is the relocation worth making. An employee who does not trust an AI decision is usually not asking for more training. They are asking what the model was allowed to decide, and in most organisations nobody has written the answer down. Publish the limits and you have converted a change-management problem into a document.
#What we got wrong
For months we treated a data-modelling problem as a performance problem. The core object in our financial intelligence platform is one company supplying another, which is a graph, and we had modelled it in whatever store happened to be nearest. Every question worth asking became a recursive query that got slower exactly as the data got more valuable. We tuned it. We changed environments. The bottleneck followed us across all of them, and the fix in the end was a data model rather than a platform. That migration is documented in full in What the Free Tier Actually Costs.
The part worth extracting here is why it survived that long. Each change produced a fresh set of numbers with nothing to hold them against: no frozen query set, no fixed corpus, no preserved before. Everything looked better in ordinary use after every change, because effort had gone into it and effort usually shows. What we never had was a comparison capable of telling us that the improvement was real but the diagnosis was wrong.
The mechanism is not carelessness. Baseline capture competes for time with the fix itself, at the exact moment the team is most certain the fix will work, and certainty is what removes the incentive to measure. That is also how organisations end up reporting AI gains nobody can verify: one justified skipped baseline at a time, until a board asks for the number and the only honest answer left is a recollection.
The remedy is procedural rather than technical. A frozen input sample is captured when a use case is approved, not when it is optimised, and it lives with the use case for as long as the use case does.
#Before you believe your own AI number
| Question | If yes | If no |
|---|---|---|
| Do you hold inputs from before the change, and can you re-run them? | You can produce a number | You have a recollection; capture the sample today |
| Would anyone outside the delivery team notice if it were switched off for a week? | It is deployed | It is a pilot with a production URL |
| Does the bill move with volume, and do you know the shape of the curve? | You can forecast | You are carrying unpriced exposure |
| At what exchange rate does this use case stop clearing its hurdle? | Price it and hedge it | The return is a currency bet nobody has authorised |
| Could you change providers with a configuration change this week? | Your exit cost is known | The vendor sets your future ROI, not you |
| Is there a written list of what the model may never decide? | You can defend it to a regulator or a partner | The limit is whatever the last engineer assumed |
The row most teams skip is the exchange rate one, and it is the row that decides whether the others matter.
#When to skip this
- You are a listed firm with a portfolio of use cases and a real capital allocation process. PwC's advice is aimed at you and lands cleanly. Portfolio discipline beats measurement rigour when you already have thirty bets running.
- You are still looking for product-market fit. Baselines, holdouts and frozen samples are overhead on a system whose shape will change next month. Ship, watch, and start measuring when the use case stops moving.
- Your inference spend is small next to payroll. The currency term is real but immaterial. Price it once, then stop thinking about it.
- The use case is genuinely new revenue rather than a substitution. There is no clean baseline to capture, because there is no before. Say that plainly rather than manufacturing a comparison.
- Our own evidence has limits. This is drawn from a handful of production systems with no control group. It is a different class of evidence from a survey of 1,217 executives, with its own standards, and it does not license a claim about what the average African company should do.
#Key takeaways
- Capture the frozen input sample when the use case is approved, not when it is optimised. The before disappears quietly and permanently.
- Set the latency ceiling before choosing the architecture. A threshold converts an improvement into a claim somebody else can check.
- Report the calls you never made as a return, in the same paper as the calls you did, or your best cost decision will read as underinvestment.
- Price every metered use case at a stressed exchange rate and state the breakeven. An unhedged AI budget in local-currency revenue is a position, not a plan.
- Keep the extraction contract identical across providers, so the exit cost stays knowable while you still have the option to use it.
- Publish what the model is not allowed to decide, and treat that document as the trust intervention. Employees withholding trust are asking about limits, not asking for training.
- Before funding a convergence play, name the data-sharing rail it depends on and check that it exists. If it does not, you are funding a partnership team to discover a missing standard.
- Treat any AI return figure without a baseline, a comparison and an outside confirmation as a belief. It may well be a correct belief. It is not a number.
Drawn from building and running AI systems in production across financial intelligence, multi-tenant publishing and programme delivery, in markets where inference is billed in dollars and revenue is not.