The first half of this series is about whether an agent does the right thing. The second half is about what it costs to run. They are usually treated as separate concerns owned by separate people, and that separation is where production readiness quietly fails.
An agent that produces excellent output at a cost that does not survive volume is not ready. An agent that is cheap and not trusted is not ready either. Neither team is wrong about their half.
Four questions, four owners, one system
Each quadrant is a different question with a different mechanism behind it.
Quality: is it doing the right thing? Answered by evals, curated examples and human review. This is the half most teams build first and the half most likely to be mistaken for the whole.
Cost: can we afford it at volume? Answered by per-turn telemetry, session discipline and ceilings. The critical qualifier is at volume. A pilot with fifteen users tells you almost nothing about the economics at fifteen hundred, particularly given how much of the cost sits in the tail of the distribution.
Observability: can we see why it behaved this way? Answered by traces that capture sources used, tools called and decisions taken. This is the quadrant that makes the other three debuggable. Without it, a quality problem and a cost problem look identical from the outside: something is wrong and nobody can say what.
Governance: who answers for the outcome? Answered by a named owner, an approval step and an audit trail. Not a committee and not a process. A person, because that is the only formulation that survives an actual incident.
What each missing quadrant produces
The failures are specific enough to name, which is useful when arguing for the work.
Every one of these is a system that demonstrated well. None of them are model problems. They are the four ways an AI project ends up quietly shelved after a successful pilot, and each of them was predictable at design time.
Why cost belongs in the readiness conversation
There is a tendency to treat cost as an operational detail to sort out after the capability is proven. In AI workflows that sequencing does not hold, for a reason specific to how these systems scale.
Traditional software has broadly predictable marginal cost. Once it works, serving it to ten times as many users is a capacity problem with known economics. An agent workflow does not behave that way. Cost scales with context, and context scales with usage patterns you cannot observe in a pilot: longer conversations, more documents, more back-and-forth, more retries on inputs the pilot never contained.
That is why instrumentation belongs early rather than after launch. Not because cost is the most important thing, but because it is the thing you cannot retrofit an understanding of. The pilot is the only time you can establish a baseline cheaply.
Why evals belong in the cost conversation
The reverse also holds, and it is less often said.
You cannot make a cost decision without a quality measure. "Can we use a smaller model here" is unanswerable in the absence of a way to tell whether the output got worse. Teams without evals make model decisions on impressions, which means they either never change anything, or change it and find out from a user.
An eval suite converts a cost question into an experiment. Run the cheaper configuration against the known cases, compare the pass rate, decide with evidence. That is a fifteen-minute exercise for a team that has the suite and an unbounded argument for a team that does not.
So the two halves are not merely both necessary. Each one is what makes the other's decisions possible.
The demo that wins nothing
Worth ending on the practical version of the argument.
The agent that gets adopted is not the most impressive one. It is the one where somebody can answer, on the day it matters, four questions: is this right, what does it cost, why did it do that, and who is accountable. A team that can answer all four gets to put a system into a real workflow. A team that can answer three gets a pilot that everyone agreed was promising.
We have made a related argument about the trust checklist for AI-built software, which covers the same gap from the delivery side.
What good looks like
All four quadrants owned, with a named person on each.
Cost baselined during the pilot, not after launch.
An eval suite that exists before the first model or cost decision is taken.
Traces detailed enough that a quality problem and a cost problem can be told apart.
A single accountable owner for the outcome, written down.
Practical checklist
- Assign an owner to each of the four quadrants before build
- Establish a cost baseline during the pilot, at realistic session lengths
- Build the eval suite before the first cost optimisation
- Capture traces that record sources, tools and decisions
- Test every proposed cost change against the eval set
- Name the person accountable for the outcome, not the committee
How CloudNala can help
We assess agent readiness across all four quadrants rather than on output quality alone, because the failures we are most often called in to fix are cost and accountability failures on systems that worked perfectly well. In practice that means baselining cost during the pilot, making sure an eval set exists before anyone starts optimising, and getting a name against the outcome while that is still an easy conversation.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Readiness Workshop or write to us at consult@cloudnala.co.za