There is a particular kind of slide that appears in AI updates to executive committees. It shows a line going up and to the right. The line is active users, or prompts per month, or tokens consumed. Sometimes it is all three. The message is that adoption is strong and the investment is working.
Half of that message is supported by the slide. Adoption may well be strong. Whether the investment is working is a different question that the slide does not address. Rising consumption means people are using the tools. It does not mean the organisation is serving customers better, deciding more accurately, losing less money or growing faster.
CloudNala's position is direct: consumption is an input to value, not evidence of it. A usage dashboard is useful for managing adoption, licences and cost. It becomes misleading the moment it is presented as a value report. This part explains why the confusion is so persistent and how to fix the dashboard without throwing away what it measures well.
Indigo Mutual is fictional, and its systems and figures are illustrative. In Part 1 its CFO asked which of fourteen AI use cases had paid off, and received a usage dashboard in reply. This part looks at that dashboard.
Why usage feels like value
The confusion is understandable. In consumer technology, usage really is a strong signal: if people keep coming back to an app they chose freely, it is probably giving them something. Enterprise AI borrows that intuition.
Inside an organisation, it does not transfer cleanly. Staff may use an AI assistant because it was rolled out to them, because their manager asked them to, because it is genuinely helpful for drafting emails, or because the alternative tool was removed. Each of those produces the same usage curve. Only one of them says much about value, and even that one says nothing about whether the time saved went anywhere useful.
There is also a commercial reason. Usage is what vendors measure, report and bill for. Their dashboards show consumption because consumption is what they sell. When an organisation adopts a vendor's dashboard as its own management report, it adopts the vendor's definition of success. The Why Do We Keep Paying for AI? series explains the two pricing models, per seat and per use. Both are measures of consumption.
Usage and value are different axes
The simplest correction is to stop drawing usage and value on the same axis.
Plot each use case on two dimensions: how heavily it is used, and how strong the evidence of value is. Four quadrants emerge, and each one calls for a different decision.
High use, strong evidence. Indigo Mutual's claims data extraction sits here. Assessors use it on most claims, and Part 3's journey instrumentation shows faster cycle times on the document types it handles well. This is where to scale.
Low use, strong evidence. The underwriting assistant helps the underwriters who use it, measurably, but only a third of them do. The problem is adoption, not the tool. Fix training, workflow integration or access before concluding anything.
Low use, weak evidence. A marketing copy tool bought by one team, lightly used and with no observable effect. Stop it or redesign it around a sharper problem.
High use, weak evidence. This is the expensive blind spot. Indigo Mutual's staff productivity assistant has 3,900 active users and is the largest single line in its AI budget. Nobody can say what it has changed. It may be valuable. It may be a pleasant convenience. The usage dashboard makes it look like the portfolio's biggest success, which is exactly why it is never questioned.
The bottom-right quadrant is where most enterprise AI consumption sits in practice, because broad productivity tools are cheap to roll out and hard to measure. That does not make them bad investments. It makes them unexamined ones.
What an AI usage dashboard is good for
None of this means usage dashboards should be thrown away. They answer real questions:
- Licence utilisation. Are we paying for 4,500 seats when 3,900 people use them? The 600-seat gap is a direct saving.
- Adoption health. Which teams have stopped using a tool after the first month? That is often the earliest warning that a tool does not fit the work.
- Model mix and cost. Which models are being called, by which applications, at what cost? This feeds the cost side of every value calculation, as covered in AI cost observability needs three views.
- Security and policy. Are people pasting sensitive data into tools not approved for it? Are unapproved tools appearing in network logs?
- Shadow AI. Which business units are buying AI subscriptions outside the approved estate?
These are all management questions about the AI estate itself. They are worth answering every month. They are just not answers to the board's question about value.
Rewrite the dashboard, do not replace it
The fix Indigo Mutual adopted was to keep every number on its usage dashboard but change what it is for.
The left column stays, but it moves down the page and becomes an operational report for the AI platform owner. The board view leads with the right column: the outcome each use case was funded to produce, cost per accepted outcome, how confident the organisation is in the evidence, risk events and the next decision with its owner.
The important move is in the bottom row of the diagram. Consumption is not discarded; it becomes the denominator. Tokens and licences feed the cost side of "cost per accepted outcome". Active users feed the adoption link in the value chain. They contribute to the value measure instead of standing in for it.
Three questions to ask of any AI usage figure
When a usage figure appears in an executive report, three questions usually reveal whether it is being used honestly:
- Compared with what? Usage of a new tool always rises from zero. The meaningful comparison is with the eligible population, with the target in the business case, or with the same period last quarter.
- And then what happened? If 3,900 people used the assistant, what changed in the work they produce? If nobody can say, the figure is adoption, not impact.
- What does it cost per useful result? If the cost cannot be related to an outcome, the figure is a bill, not a benefit.
These are not hostile questions. A team that can answer them is managing its use case well. A team that cannot is not failing; it simply has not been asked to design for value yet, and Parts 2 and 3 show how.
When consumption is the right measure
There is one honest exception. For low-cost, broad productivity tools where the per-person price is small and the benefit is diffuse, it may not be worth building rigorous measurement. A board can reasonably decide to treat such tools like email or spreadsheets: a cost of doing business, managed for utilisation and security rather than for proven return.
That is a legitimate decision. What matters is that it is made explicitly, recorded as such and reflected in how the tool is reported. The problem is not consumption metrics. The problem is consumption metrics wearing the clothes of value. So which AI should we actually pay for? offers a simple way to decide which tools deserve which level of scrutiny.
Next: Part 5, What an AI value dashboard should show the board. All parts are on the series page.
How CloudNala can help
We review existing AI dashboards and portfolio reports, separate the consumption measures from the value measures, and redesign the board view so usage feeds cost per outcome rather than replacing it. Where the evidence of value is missing, we help the teams concerned decide whether to instrument the use case properly or to manage it explicitly as a cost of doing business.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Value Review or write to us at consult@cloudnala.co.za