Architecture teams record big decisions. Which cloud, which database, whether to build or buy. Those go into architecture decision records, review boards and design documents.
Upgrades do not. A gateway moving from one minor version to the next, a model identifier changing, an orchestration framework picking up a new major release: these happen in pull requests and chat threads. Six months later nobody can say why a version was chosen, what was tested before it shipped or what the previous version was.
For most libraries that is tolerable. For shared AI infrastructure it is not, because an upgrade can change cost, behaviour, security posture and data handling in one step, and because the first question in any incident is "what changed?"
Langeberg Software is fictional, and its systems and figures are illustrative. When Langeberg worked through the incident in Part 3, the production question was answered quickly because the gateway was pinned. A different question took a day: why was production on that particular version, and was it safe to move forward? Nobody had written it down.
The upgrade record
Langeberg's answer is an upgrade record: eight fields, written once per upgrade of a shared AI component, stored in the same repository as the architecture decision records.
- What changed. Component, from version, to version, with digests.
- From where. Source, whether the release was signed, how the artefact was verified.
- Why now. The reason: a fix, a feature, a security advisory, an end-of-support date.
- Tests run. Contract tests, the evaluation suite, load or soak results, with links.
- Security review. Release notes read, new permissions or network calls, dependency changes, known advisories.
- Compatibility. Configuration changes, behaviour changes, anything consumers must do.
- Rollback plan. The previous digest and configuration, and confirmation that rollback has been tested for this change.
- Decision. Who approved it, and when.
The test for a good record is simple: six months later, can someone who was not there tell what ran, why it changed and how to go back?
Why the evaluation result belongs here
For ordinary software, "tests passed" is usually enough. For AI infrastructure it is not, because upgrades change behaviour in ways unit tests do not see.
A gateway release can change retry defaults, timeout handling, streaming behaviour or how it formats requests for a provider. An orchestration framework can change how it assembles prompts or handles tool errors. A model identifier change can alter answers across the board. None of these breaks a contract test. All of them can change what customers see.
So the record links the evaluation suite's result for the upgrade, compared with the previous version. When Langeberg moved its gateway between minor versions, the evaluation run showed a small rise in invoice extraction failures on long documents, traced to a changed default timeout. The fix was one configuration line. Without the comparison, it would have reached customers as an intermittent defect. Evals that gate real work covers how to make that kind of gate meaningful.
The record is an input, not an archive
Records that are written and never read are paperwork. These get read, at three predictable moments.
At the next upgrade. The previous record says what was tested last time, what broke, what configuration is load-bearing. The next upgrade starts from there instead of from scratch.
During an incident. When an advisory names a version, the records answer "were we ever on it?" and "when did we move?" from a file. When behaviour changes unexpectedly, the most recent record is the first suspect.
When someone asks. Auditors, customers in procurement and new team members all ask versions of "how do you manage changes to this?" A folder of upgrade records is the answer.
Model changes are upgrades too
The record is easy to apply to a gateway or a framework, because they have version numbers and release notes. Model changes are easier to miss, and they deserve the same treatment.
A hosted model identifier changing from one dated version to the next is an upgrade. So is a provider retiring a model and moving traffic to its successor, or a routing rule sending a share of requests to a cheaper model. Each one can change answers, latency and cost at once, with no code change anywhere.
For these, Langeberg adapts three of the fields:
- From where records the provider, the exact model identifier and the region, because the same model name can be offered under different data-processing terms in different regions.
- Tests run always includes an evaluation comparison against the current model on the application's own evaluation set, plus a cost comparison on a representative week of traffic.
- Rollback plan notes how long the previous model will remain available. Providers announce retirement dates, and a rollback plan that points at a retired model is not a plan.
Provider retirement notices also become a trigger. When one arrives, the team opens an upgrade record that day, with the retirement date as its deadline, rather than discovering the change when calls start failing. The Economics of AI Delivery series covers how to choose between models on quality and cost; the record is where that choice is written down.
Keep it proportionate
The fastest way to kill this practice is to make every record a document. Three rules keep it light.
Only for shared or high-reach components. The gateway, orchestration frameworks, agent runtimes, production model identifiers, MCP servers with credentials. Not every library bump in every service.
Generate what can be generated. Versions, digests, test links and the evaluation comparison can be filled in by the pipeline. A person writes "why now", reviews the security notes and makes the decision.
Template in the pull request. The upgrade's pull request carries the record as its description. Merging it files the record. No separate system.
At Langeberg a typical record takes about fifteen minutes to complete beyond the work the upgrade already required. The first incident it shortened paid for a year of them.
Where it sits in the architecture record
Upgrade records are the small, frequent decisions that sit beside the large, rare ones. Together with the AI bill of materials from Part 6, they make the architecture record describe the system as it is now, and how it got there, rather than as it was designed two years ago. That is the same argument made in The Architecture Is Not the Diagram: the decisions and their evidence are the architecture; the picture is a view of them. The Architecture in Plain English series picks up the idea of architecture artefacts carrying their own evidence.
Next: Part 9, Open source does not mean low risk or high risk. All parts are on the series page.
How CloudNala can help
We help platform and architecture teams set up a lightweight upgrade record for shared AI infrastructure, wired into the pull request and pipeline so that most of it fills itself in, with evaluation comparisons and rollback evidence attached. The aim is a record people actually read at the next upgrade and the next incident.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Supply Chain Review or write to us at consult@cloudnala.co.za