An engineering team adopts a coding agent. Productivity is up, everyone is pleased, and the first serious invoice arrives. The immediate reaction is almost always the same: which model is doing this, and can we use a cheaper one?
That instinct is reasonable and frequently wrong. A useful piece of work by Andrew Baker on instrumenting coding agents, published in August 2026, makes the case with real numbers, and the short version is that model price was not the binding constraint. Session behaviour was.
Why the second row costs so much more
Coding agents work by carrying the conversation forward. Each turn sends the accumulated context back to the model so it can reason about what has happened so far. That is what makes them useful: the agent remembers the file it read forty minutes ago.
It also means the cost of a turn is not determined by what you just typed. It is determined by everything the session has accumulated up to that point. A short exchange late in a very long session is not a cheap request.
In the workload Baker instrumented, context grew from under 40,000 tokens to over 900,000 across roughly 6,000 turns in a session that ran for about a week. Every turn near the end carried close to a million tokens of history. The model price per token never changed. The volume did, continuously, in one direction.
The courier analogy
For a non-technical stakeholder, this is the version that lands.
You are paying a courier to deliver a one-page answer. Reasonable cost. But the courier's standing instruction is to bring along every document produced on this account so far, so the one-page answer travels with a 900-page appendix. You have negotiated a good rate on the appendix, because it is the same one every time and the courier has it on the van already. It is still on the van. You are still paying for the trip.
The negotiation on the appendix rate is caching, and it is worth having. It is not the same as asking whether the appendix needs to go at all.
What teams check first, and what usually explains it
The left column is not wrong. Model choice matters, and there are workloads where moving a class of task to a smaller model is exactly right. The problem is sequencing. Changing models before measuring context means you are making a large change to output quality in order to address a cost driver you have not confirmed.
The common outcome is that quality drops, the team notices, they move back, and the bill is unchanged, because the bill was never about the model.
The distribution matters more than the average
One further finding from the same investigation is worth internalising, because it changes what you do about the problem.
The top ten percent of sessions accounted for 63.2 percent of the spend. That is a power-law distribution, not a normal one, and it means the average session tells you almost nothing useful. Optimising the typical session gets you very little. Finding the handful of pathological sessions gets you most of the available saving.
This is good news operationally. You do not need an organisation-wide behaviour change programme. You need to identify perhaps a dozen sessions a month and understand what happened in them, which is a small enough task that one person can do it in an hour.
It also means an averaged dashboard will actively mislead you. Average cost per session, average tokens per session, average anything: all of these will look stable while the tail does the damage. The reporting has to preserve the distribution, which is a point we return to in the weekly metrics article.
The cheapest model is not always the cheapest workflow
Worth stating as a principle, because it generalises past coding agents.
A smaller model that needs three attempts to produce an acceptable result is not cheaper than a larger model that needs one, once you count the context each attempt carried. A cheaper model that produces work requiring more human review has moved cost from a line item you can see to one you cannot. And a model change that lengthens sessions, because the agent is less able to finish a task cleanly, can increase total cost while reducing per-token cost.
The unit that matters is cost per accepted unit of work, not cost per token or per session. Nothing else survives contact with the question "did we get cheaper or just worse".
What to do before touching the model
Measure first. That is genuinely the whole recommendation at this stage, and it is the subject of the instrumentation article.
Specifically, find out how long your sessions run, how much context they accumulate, and which ten sessions cost the most last month. If those three numbers are unremarkable, the model conversation is the right one to have. If sessions are running for days and carrying hundreds of thousands of tokens, you have found the lever, and it costs nothing to pull.
Practical checklist
- Do not change models before measuring session length and context growth
- Find the top ten sessions by cost from last month and read them
- Report distributions, not averages
- Track cost per accepted unit of work, not cost per token
- Establish how long a typical session runs before deciding anything
- Treat a long-running session as a cost event, not a workflow preference
How CloudNala can help
The first engagement here is usually a measurement pass rather than an optimisation one: get per-session and per-turn data out of the tools already in use, look at the distribution rather than the average, and read the handful of sessions that dominate the bill. That is often enough to make the right change obvious, and it avoids the common and expensive detour of downgrading models to solve a problem models were not causing.
Andrew Baker's article on instrumenting OpenCode and Claude Code is the source for the figures cited here and is worth reading directly if you are setting this up.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Readiness Workshop or write to us at consult@cloudnala.co.za