Context trimming without losing the evidence

Tugela's triage agent got cheaper in September and worse in October, and for three weeks nobody connected the two. The engineering team had cut the average input of each claim run by more than half. The monthly invoice fell accordingly. Then an assessor noticed that the agent had recommended fast-track settlement on a household contents claim where the policy schedule carried an endorsement excluding theft from unattended vehicles. The endorsement had been in the claim file. It had not been in the agent's context, because a summarisation step had compressed the schedule to "standard contents cover, R180,000 sum insured".

The saving was real. So was the error. The two came from the same decision, made without asking which parts of the context were evidence for the recommendation and which were just bulk.

Tugela Claims Services is fictional, and its systems and figures are illustrative. Tugela is the Pietermaritzburg claims administrator that has carried this series: about 18,000 short-term claims a month for seven insurers, and a triage agent that drafts a routing recommendation for one of 26 assessors to accept or change. This part adds a mechanism the earlier parts assumed: how to make context smaller without making the decision weaker.

Why trimming is tempting, and where it goes wrong

Part 1 explained that input tokens are metered whether or not they help. Long claim files, full policy wordings, conversation history and verbose tool results all add input. Trimming is the obvious lever because it needs no new model, no new vendor and no renegotiation. A developer can halve the input of a run in an afternoon.

The trouble is that most trimming techniques are blind to meaning. Truncating the oldest messages, keeping only the last few tool results or asking a model to summarise everything treats every token as equally disposable. In a claims workflow they are not. A sentence in a policy endorsement can decide the outcome. A paragraph of email pleasantries cannot.

There is also a quality reason to trim, not only a cost reason. Research on long-context behaviour, notably Lost in the Middle (Liu and colleagues, Transactions of the ACL), found that models used information best when it sat near the beginning or end of the input and worse when it sat in the middle of a long context. Anthropic's engineering guidance on context engineering (September 2025) makes a related point about recall falling as context grows. So a well-trimmed context can be both cheaper and better. A badly trimmed one is cheaper and worse, and the cost dashboard cannot tell the difference.

Three classes of context

The fix starts with classification. Before choosing a trimming technique, decide what each piece of context is for. Tugela uses three classes.

Evidence is any content the recommendation may rely on and that an assessor, auditor or complainant may later ask to see: policy wording and endorsements, the schedule, the claimant's statement, assessor notes, prior-claim records, the result of a fraud check. Evidence is kept verbatim or not at all. It is never paraphrased into a summary, because a summary of a clause is a new clause written by a model.

Working state is what the agent needs to keep track of the task: which documents it has read, which checks are complete, what it is still unsure about. Working state can be summarised, provided the summary points back to the evidence it came from.

Noise is content that neither supports the decision nor tracks progress: duplicate attachments, email signatures, raw tool output already reduced to a result, retries of the same lookup. Noise can be dropped, and dropping it is where most of the saving usually is.

The classification is a design decision, not a model decision. It belongs in the workflow's configuration, owned by the people who answer for the outcome. For Tugela that is the claims operations manager, with the engineering lead advising on cost.

Trim each class differently

Once content is classified, each class gets its own technique.

Two of these deserve more detail.

Select evidence, do not compress it. The cheapest way to carry less evidence is to carry only the evidence that matters for this claim. A motor windscreen claim does not need the household contents wording. Within a policy, the agent needs the sections that apply to the loss type, plus every endorsement, because endorsements are where exceptions live. That is a retrieval problem, and Reranking, context budgets and cost covers the retrieval side. The rule added here is simple: endorsements and exclusions are always selected, whatever their similarity score.

Reference evidence the agent may need later. Instead of carrying the full prior-claims history, the agent receives an index: claim numbers, dates, loss types and a one-line outcome, each with an identifier. If the agent decides a prior claim is relevant, it fetches that record by identifier. The record arrives verbatim. The index is working state; the fetched record is evidence.

Protect evidence from automatic clearing

Most agent frameworks and model platforms now offer some form of automatic context management. As a dated example, Anthropic documents a context editing capability that clears older tool results once the input passes a threshold, keeps a set number of recent tool uses, and accepts an exclude_tools list of tools whose results are never cleared. It is in beta, enabled by a beta header, and the documentation notes that clearing invalidates cached prompt prefixes (checked on 5 November 2026). Other platforms offer comparable features under different names.

Features like this are useful, and the exclusion list is the part that matters for evidence. At Tugela, the tools that return policy wording, endorsements and the fraud-check result are excluded from clearing. The tools that return raw document text after extraction, search results and address lookups are not. The decision about which tools are excluded is recorded in the same configuration as the context classes, so a platform default never quietly decides what counts as evidence.

The caching interaction matters too. Part 4 showed that caching rewards a stable prompt prefix. Aggressive, frequent clearing breaks that prefix and can turn a cache saving into a cache-write cost. Trim in larger, less frequent steps, and put stable evidence early in the prompt so it survives both clearing and caching. Where a team reports a high cache hit rate alongside a large context, the hit rate may be discounting waste rather than proving efficiency.

Summaries need receipts

Summarisation is still useful for working state, and for long-running agents it is unavoidable. The failure at Tugela was not that a summary existed. It was that the summary had no receipt: nothing recorded what it replaced, and nothing in it pointed back to the source.

A summary with a receipt has three properties:

• Pointers. Each statement in the summary carries the identifier of the source it came from, so the agent or a reviewer can fetch the original.
• A trim log. The run record notes what was removed or summarised, when, by which rule, and how many tokens it saved. This is cheap to write and decisive in a complaint, because it shows whether the agent ever saw the clause in question.
• Open questions preserved. Anything the agent was unsure about survives the summary as an explicit open item, not as a smoothed-over conclusion.

The trim log also answers the question every finance director eventually asks: what did the saving cost? If the log shows that endorsements were summarised on 4% of runs, the team knows which recommendations to re-check. The prompt is part of the evidence argued that what a model saw belongs in the record, and Conversation history is not a memory strategy explains why long-running agents need deliberate summaries rather than an ever-growing transcript. A trim log is how you keep that promise once context is no longer static.

Evaluate the trimming, not only the answer

Part 3 argued that a cheaper model is acceptable only if it passes your own evaluation. The same rule applies to a cheaper context. Any change to trimming rules is a change to the system and should run against the evaluation set before release.

Tugela added two checks to its existing triage evaluation:

1. Evidence recall. For each case in the evaluation set, assessors marked the clauses and records a correct recommendation must rely on. The check measures whether those items were present, verbatim, in the context the agent actually saw. Replaying the earlier configuration against the same cases showed that, before the September change, recall on endorsements had been complete. After it, it had fallen sharply on household claims, which is where schedules carried the most endorsements.
2. Outcome agreement. The usual check: does the recommendation match the assessor-agreed outcome? This check fell only slightly after the change, because most claims do not turn on an endorsement. That is exactly why it missed the problem. An aggregate accuracy figure can absorb a serious error that only affects a minority of cases.

Evidence recall is cheap to compute, because it does not need a model to judge anything. It is a lookup: was the marked text in the context or not. It is the single most useful test a team can add before touching context size.

What Tugela changed, and what it cost

Tugela did not reverse the trimming. It reclassified.

• Policy wording is now selected by loss type, with every endorsement and exclusion always included verbatim.
• The prior-claims history is an index with fetch-by-identifier, not a block of text.
• Raw extracted document text is cleared after the agent records the facts it found, each with a page reference.
• Email threads are deduplicated and signatures dropped before the agent sees them.
• Summaries of working state carry source pointers, and every run writes a trim log.

The average input per run rose by about a fifth from its September low, and was still well under half of the original. In the series' preferred unit, cost per accepted outcome, the result was better than September's, because assessors stopped overturning fast-track recommendations on endorsed policies. A cheap recommendation an assessor rejects is the most expensive kind.

A trimming checklist

Before any change that reduces context, ask:

• Is every piece of context classified as evidence, working state or noise, and is the classification owned by the business owner of the decision?
• Is evidence selected rather than compressed? No paraphrased clauses, no truncated documents.
• Are exceptions always selected? Endorsements, exclusions, flags and prior adverse findings, regardless of similarity scores.
• Are evidence-returning tools excluded from any automatic clearing, by explicit configuration?
• Does every summary carry pointers, and does every run write a trim log?
• Does the evaluation measure evidence recall, not only outcome agreement?
• Has the caching effect been checked, so trimming does not destroy a cache saving?
• Is the saving reported per accepted outcome, not per run?

Key points

• Trimming is the fastest way to reduce AI run cost and the fastest way to remove the evidence a decision depended on.
• Classify context as evidence, working state or noise, and trim each class differently: select evidence verbatim, summarise working state with pointers, drop noise.
• Exclude evidence-returning tools from automatic context clearing by explicit configuration, and remember that clearing interacts with caching.
• Give every summary a receipt: source pointers, a trim log and preserved open questions.
• Add an evidence-recall check to the evaluation, because aggregate accuracy can hide errors that affect only the cases with exceptions.

This is the final part of The Economics of AI Delivery. Start again from Part 1, A token is a unit of compute, not a unit of value. For the memory side of the same problem, see Conversation history is not a memory strategy.

How CloudNala can help

CloudNala reviews the context design of AI workflows before and after cost reduction work. We classify the context with the business owner, set evidence-selection and exclusion rules, add evidence-recall checks to your evaluation, and report the result as cost per accepted outcome, so that the saving you bank is not paid back later in overturned decisions and complaints.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Cost Review or write to us at consult@cloudnala.co.za