Cost per prompt is the wrong metric: measure cost per accepted outcome

The first AI cost metric most organisations produce is cost per prompt, or its close relative, cost per conversation. It is easy to calculate: take the monthly invoice and divide it by the number of requests users made. It is also nearly useless for decisions, because both halves of the division are wrong.

The top half is wrong because the invoice is only part of what the work cost. The bottom half is wrong because a prompt is not a unit of work. It is a unit of asking.

This part replaces the metric with one that survives contact with finance: cost per accepted outcome.

Tugela Claims Services is fictional, and its systems and figures are illustrative. Tugela's triage agent, introduced in Part 1, drafts a routing recommendation for each new short-term insurance claim. An assessor accepts it, changes it or discards it and handles the claim manually. Tugela uses an exchange rate of R18 to the dollar in these examples.

One request, many metered events

When a Tugela assessor opens a claim and asks for triage, the dashboard records one request. The provider records something quite different.

The agent reads the claim. It retrieves the relevant policy section, then retrieves a second section when the first does not cover the loss type. It plans what to check. It calls three tools: the policy administration system for cover dates, the claims history service for prior claims, and a vehicle valuation lookup. It drafts a routing recommendation. Then it runs a self-check against the triage rules and, if the check passes, returns the result.

That is nine model calls. Each one carries the instructions, tool definitions and everything gathered so far, so the input grows as the run progresses. In Tugela's measurements the typical run sends about 96,000 input tokens and receives about 7,500 output tokens across those nine calls. At the list prices used in Part 1, that is roughly $0.27, or R4.81, per run.

The point is not the specific number. It is that the cost of a task is a property of the agent's behaviour, not of the user's request. A difficult claim that sends the agent around its loop twice more costs twice as much while still appearing as one request on the dashboard. The earlier article Observability and cost shows how difficult cases skew the average; this part is about what to divide by.

What belongs in the numerator

Cost per accepted outcome starts with everything it took to produce the results, not only what the provider billed. For an AI workflow with human review, there are usually four lines.

Model and tool calls. Every run, including the ones whose output was later discarded. Discarded runs are not free because nobody used them.

Human review. If an assessor spends three minutes checking each recommendation, that time is part of the cost of the AI path. It is often the largest line. Tugela values assessor time at a loaded cost of R350 an hour, so three minutes is R17.50 per case.

Rework. When a recommendation is rejected, someone handles the claim the old way. That time would have been spent anyway without the agent, but in the AI path it is spent after the agent's cost has already been incurred. Tugela measured about 25 minutes of assessor time per rejected case.

Platform and assurance. Evaluation runs, monitoring, the engineering time that keeps the agent working, and any third-party services. These are usually monthly fixed costs; spread them over monthly volume.

What belongs in the denominator

The denominator is the number of outcomes that were actually used. That needs a definition per workflow, agreed with the people who own the outcome.

For Tugela, an accepted outcome is a recommendation the assessor applied without changing the routing. A recommendation where the assessor changed the routing counts as rejected, even if the justification text was useful. That is a deliberately strict definition. A looser one, counting partial acceptance, would flatter the number and hide the cases where the agent was close but wrong.

Other workflows need their own rule:

Doing the division

Take one hundred Tugela triage runs.

Model calls cost R481 across all one hundred runs. Assessor review at three minutes each cost R1,750. Eighteen runs were rejected and needed about 25 minutes of manual handling each, which is R2,625. Leaving platform cost aside for a moment, the AI path cost R4,856 for one hundred claims.

Divided by one hundred prompts, that is R48.56. Divided by 82 accepted recommendations, it is R59.22. And the figure most dashboards would show, the model cost alone divided by prompts, is R4.81, about a twelfth of the real cost of a useful outcome.

Three conclusions follow from this example, and they generalise.

1. The model is rarely the largest cost line in a human-reviewed workflow. Optimising tokens while ignoring review time is optimising the smallest number on the page.
2. Acceptance rate is a cost lever. Raising acceptance from 82% to 90% changes cost per accepted outcome more than halving the model price would, because each rejection carries 25 minutes of rework.
3. Review time is a design decision. If the recommendation is laid out so that an assessor can verify it in one minute instead of three, the AI path gets cheaper without touching the model. Showing the evidence the agent relied on, as discussed in The human in the loop must show their work, is a cost control as well as a governance one.

Comparing against the path without AI

Cost per accepted outcome becomes a business case only when it is compared with the cost of the same outcome without the agent. Before the agent, Tugela's assessors spent an average of 12 minutes triaging a claim, which is R70 at the loaded rate. The AI path's R59.22 per accepted outcome is lower, but not by the margin the original pitch implied, and the margin depends heavily on acceptance rate.

That is a more honest business case than "the model costs R4.81 a claim". It also points at where to spend improvement effort: not on a cheaper model, but on raising acceptance and shortening review.

Building the measurement

None of this needs a large platform. It needs four fields on every run, recorded consistently:

• a case identifier that ties every model call to the claim;
• the token counts and price for each call, which the provider already returns;
• the review outcome (accepted, changed, discarded) captured where the assessor already works;
• the review duration, which most case-management systems can timestamp.

Platform cost can be allocated monthly. Rework can be estimated from a sample at first and measured properly later. The weekly metrics in Four AI engineering metrics teams should measure every week are a good model for the cadence, even though they were written for coding agents.

What matters most is that the acceptance definition is agreed with the business owner before anyone looks at the number. Once a number exists, there is always pressure to redefine success so that it looks better.

Next: Part 3, Use the smallest model that reliably passes the evaluation. All parts are on the series page.

How CloudNala can help

We help teams define acceptance for each AI workflow with the people who own the outcome, instrument runs so that model cost, review time and rework land against the same case, and build the comparison with the non-AI path. That turns an invoice into a unit cost the business can make decisions with.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Cost Review or write to us at consult@cloudnala.co.za