Every AI service that bills by use counts the same thing: tokens. The invoice counts them, the provider console counts them, and before long the internal dashboard counts them too. A month later someone presents a chart showing token consumption up 40% and asks whether that is good or bad.
Nobody in the room can answer, because the chart measures the wrong thing for the question. A token is a unit of compute. It tells you how much reading and writing a model did. It does not tell you whether that reading and writing produced anything anyone needed. Two pieces of work with identical token counts can be worth nothing and worth a great deal.
This series is about the economics of AI delivery: how to measure, govern and explain the variable cost of AI-assisted work so that it can be compared with what the work achieves. It sits alongside two earlier bodies of work on CloudNala Insights. Why Do We Keep Paying for AI? explains the purchase decisions, seats against usage and reserved capacity. The second half of Building Governed AI Agents explains why context volume, not model price, drives the bill for coding agents. This series picks up the questions those leave open: what an accepted outcome costs, which model tier to use, how caching and batch pricing actually work, how to quote a pilot, and how to price an AI feature you sell.
The position we take throughout is simple. Token consumption is measurable infrastructure usage. It becomes useful only when it is attributed to a task and set against quality, latency, risk and the effort it replaced. Low token use is not efficiency, and high token use is not waste. Both are just volume.
Tugela Claims Services is fictional, and its systems and figures are illustrative. Tugela is a Pietermaritzburg claims administrator that handles about 18,000 short-term insurance claims a month on behalf of seven insurers: motor, household contents and a large volume of windscreen claims. Twenty-six claims assessors decide how each claim should be handled. Over the last year Tugela built a triage agent that reads a new claim, its documents and the relevant policy wording, and drafts a routing recommendation for an assessor to accept or change. The finance director has started receiving a monthly AI invoice in dollars, and the board has started asking what it buys.
What actually gets metered
A token is a fragment of text, roughly three quarters of an English word on average. Providers count tokens in two directions and price them differently.
Input tokens are everything sent to the model on a call. In a real application, the user's words are usually the smallest part. Tugela's triage call sends the instructions that tell the agent how to behave, the definitions of the tools it may use, the claim form, any attached documents that have been converted to text, and the policy wording for the insurer concerned.
Output tokens are everything the model writes back, including any reasoning the provider bills as output. For Tugela that is a structured routing recommendation and a short justification.
The two directions have separate prices, and output is consistently the expensive one. On 27 October 2026, one major provider's published list price for its current mid-tier model was $2 per million input tokens and $10 per million output tokens, a five-to-one ratio. Its larger and smaller tiers kept the same ratio at different levels. Other providers publish different numbers, and all of them change, so treat these as a dated example rather than a reference price.
Run the numbers on one Tugela call. Fourteen thousand input tokens at $2 per million is $0.028. Twelve hundred output tokens at $10 per million is $0.012. The answer is less than a tenth of the volume and a third of the cost.
That asymmetry is the first practical lesson. Input and output are separate levers, and they respond to different design decisions.
- Input volume is driven by what you choose to send: how much policy wording, how many retrieved passages, how long the instructions are, and how much conversation history is replayed each turn. Retrieval discipline, as covered in Reranking, context budgets and cost, and caching, which Part 4 covers, act on input.
- Output volume is driven by what you ask the model to produce: free prose or a compact structure, one draft or three alternatives, visible reasoning or none. Asking for a structured recommendation instead of a narrative essay acts on output, and at a five-to-one price ratio it acts strongly.
A team that only watches the total cannot tell which lever moved.
Why token counts invite the wrong conclusions
Because tokens are the unit on the invoice, they become the unit of management by default. Three mistakes follow.
Mistake one: treating fewer tokens as efficiency. A cheap call that produces a recommendation an assessor throws away was not efficient. It was waste that happened to be small. Tugela's earliest prototype used a very short prompt and a small model, consumed a fraction of today's tokens, and was overruled on more than a third of claims. The token chart looked excellent.
Mistake two: treating more tokens as waste. A complex motor claim that requires the agent to read a police report, two quotes and a fourteen-page policy schedule will consume many more tokens than a windscreen claim. If that work catches an inconsistency that saves a fraudulent payout, the extra tokens were the best money spent all month.
Mistake three: treating token growth as a budget problem. If claims volume grows and the agent handles more of them, tokens grow. That is the thing you were hoping would happen. The question is whether cost per useful outcome is stable, falling or rising, which a total cannot show.
The quadrant above is the reason a token dashboard needs a second axis. Without one, the bottom-left box looks like success and the top-right box looks like a problem.
The unit you actually want
The economic unit for AI work is the same one any operations manager would use for people: cost per accepted outcome. For Tugela that is cost per claim correctly routed and accepted by an assessor. For a proposal team it might be cost per section approved by the bid manager. For a coding team it might be cost per change merged without rework.
Getting there requires three things that a raw token count does not provide.
- Attribution. Every model call is tagged with the task, case or work item it served. Without this, tokens are a pool rather than a cost of something. The mechanics of tagging, and the three views different audiences need, are covered in AI cost observability needs three views.
- An acceptance signal. Someone, or some deterministic check, records whether the output was used as produced, used after correction, or discarded. Tugela records the assessor's decision on every recommendation, which costs nothing because the assessor already makes that decision.
- The rest of the cost. Human review, rework when the output is rejected, evaluation runs, and the platform the agent runs on. Part 2 shows why, in Tugela's case, these are larger than the token bill.
Once those exist, tokens become an input to a meaningful number rather than a number people argue about.
Plain-English definitions worth agreeing internally
Much of the confusion in AI cost conversations comes from people using the same words for different things. These are the definitions we use with clients and in this series.
| Term | Meaning |
|---|---|
| Token | A fragment of text the model reads or writes. The unit of metering. |
| Input tokens | Everything sent to the model on one call, including instructions, tools, documents and history. |
| Output tokens | Everything the model writes back, including billed reasoning. Usually several times the input price. |
| Call | One request to a model. A single task may involve many calls. |
| Run | All the calls made to complete one task or case. |
| Accepted outcome | A run whose result was used, as judged by a person or a deterministic check. |
| Cost per accepted outcome | All costs for a set of runs, including review and rework, divided by the number accepted. |
Agreeing these before the first budget conversation saves several later ones.
Where South African budgets feel this differently
Two local factors change the arithmetic, and both get fuller treatment in Part 5.
AI services are priced in dollars and budgeted in rand, so the same token count produces a different rand cost every month. A forecast that does not carry an exchange-rate range is a forecast that will be wrong in a predictable direction about half the time. Observability and cost made this point for production agents; it applies equally to the pilot budget.
Where data residency matters, the choice of where inference runs can carry a price difference too. One provider published a 10% premium on 27 October 2026 for regional rather than global endpoints on its partner clouds, and the same uplift for restricting inference to the United States on its own platform. Whether South African data needs to stay in a particular region is a separate question, covered in POPIA does not say your data must stay in South Africa. When it does, the premium should be a known line in the business case, not a surprise.
What Tugela changed
Tugela did not cut its token usage after this conversation. It changed what it reported.
The monthly AI report to the executive committee now leads with three numbers: claims triaged, the percentage of recommendations accepted without change, and cost per accepted recommendation in rand. Token volume and the dollar invoice sit underneath, as supporting detail for the engineering team. The first month's report showed token consumption up 22% because the agent had started handling household claims as well as motor. Cost per accepted recommendation had fallen, because household claims are simpler and acceptance was high. Under the old report, that month would have produced a cost-control meeting.
The engineering team also split its tracking into input and output. That revealed that one prompt change, asking the agent to "explain your reasoning in detail" for the assessor's benefit, had nearly doubled output tokens. Assessors, it turned out, read the first two lines. The instruction now asks for three bullet points.
The series from here
Each part takes one decision a budget holder or architect will face:
- Part 2: why cost per prompt is the wrong metric, and how to build cost per accepted outcome.
- Part 3: using the smallest model that reliably passes your evaluation.
- Part 4: prompt caching as an architecture decision, with the mechanics that decide whether it pays.
- Part 5: batch versus realtime pricing, and the rand.
- Part 6: quoting the AI run cost before a pilot starts.
- Part 7: when AI tooling is a substitution and when it is just another cost.
- Part 8: pricing an AI feature you sell to your own customers.
Next: Part 2, Cost per prompt is the wrong metric. All parts are on the series page.
How CloudNala can help
We help finance and technology leaders put AI spend on a footing they can defend: agreeing the definitions, tagging model calls to the work they serve, capturing an acceptance signal, and producing a cost-per-outcome view that sits next to the provider invoice. The aim is a monthly report that answers the board's question, which is never "how many tokens" but "what did this buy".
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Cost Review or write to us at consult@cloudnala.co.za