Every AI decision rests on claims. The vendor said the product cuts handling time by forty per cent. The pilot team said users saved fifteen minutes a day. The board paper said most AI pilots fail, so the organisation should be careful. Someone said the model is accurate enough for customer-facing use.
Some of those claims are true. Some were true once. Some were never checked. In most organisations there is no way to tell which is which, because the claims were never written down as claims. They appear in slides, emails and meeting notes, shape a decision, and then disappear. Months later, when the result disappoints, nobody can say which assumption was wrong.
A claims register fixes that. It records each claim that materially influences an AI decision, in the words used, with the evidence held, a grade, an owner and an expiry date. It is a simple tool, and it ties the earlier parts of this series together. Part 1 showed how a time-saving claim can survive without evidence. Part 2 showed how widely quoted statistics need reading carefully. Part 3 showed how a public proposal can rest on goals nobody stated. In each case the problem was an unexamined claim.
Tsitsikamma Hardware Group is fictional, and its systems, claims and figures are illustrative.
Why a register of claims, not only of benefits
The Boardroom AI Test series already describes a benefits ledger: one row per use case, tracking its objective, owner, baseline, evidence and next decision. That ledger tracks outcomes. A claims register tracks something earlier and more slippery: the assertions that justified the decisions in the first place.
The two are complementary. The benefits ledger asks "did the use case deliver?" The claims register asks "what did we believe when we decided, and was it true?" One claim can support several use cases, and when it turns out to be wrong, every decision it supported needs a second look. A vendor accuracy claim that fails on the organisation's own data affects every use case built on that product. Without a register, nobody knows which ones those are.
The claims register also covers claims that never become use cases: the statistic quoted in a strategy paper, the competitor rumour that triggered a programme, the regulatory interpretation that ruled an option out.
What counts as a claim
A claim belongs in the register if it is a factual or predictive statement that has, or will, materially influence a decision about AI investment, risk or design. Four kinds come up most often:
- Vendor claims. Accuracy, time savings, cost reductions, compliance, data handling, roadmap commitments.
- Internal claims. Pilot results, time savings, user satisfaction, readiness, error rates.
- External claims. Statistics from reports, articles and analysts quoted in decision papers.
- Risk and regulatory claims. "This is allowed", "this is not personal information", "the model does not retain our data".
Opinions and preferences do not belong. "We should prioritise customer service" is a choice, not a claim. "AI will cut customer service costs by thirty per cent" is a claim.
Four evidence grades
Each claim is graded by the evidence the organisation actually holds, not by how plausible it sounds or how reputable its source is.
Asserted means someone said it. A demonstration, a sales deck, a headline or a confident colleague. An asserted claim can be true, but it cannot support a funding decision on its own.
Published means there is a documented source whose basis can be examined, such as a study or case with its method and sample stated, or binding contract terms, so its relevance can be judged. A published claim can justify a bounded pilot, if the conditions are similar enough.
Reproduced here means the organisation has tested the claim on its own data, processes and users under controlled conditions. That can justify scaling.
Observed in production means the claim has been measured in live operations against the recorded baseline. Only that grade should be reported to a board as realised value, which is the same distinction drawn in What an AI value dashboard should show the board between forecast, evidenced and verified benefits.
The grade is a ceiling, not a verdict. A claim graded Published may still be wrong for this organisation. That is why the register also records status.
The register in practice
At Tsitsikamma the register is a shared table owned by the finance team's AI value lead, with each row owned by the person whose decision depends on it. A few rows show how it works.
| ID | Claim, as worded | Source and type | Decision it supports | Grade | Status | Owner | Expires |
|---|---|---|---|---|---|---|---|
| C-014 | "Saves each user about fifteen minutes a day" | Pilot survey, internal | General assistant renewal | Reproduced here | Qualified: timed sample showed review time absorbed most of it | Head of operations | Mar 2027 |
| C-021 | "Extracts invoice fields with over 95% accuracy" | Vendor sales material | Invoice-matching build | Reproduced here | Supported for printed invoices; rejected for handwritten | AP manager | Jan 2027 |
| C-030 | "95% of generative AI pilots fail" | MIT NANDA report via press coverage, external | Strategy paper caution | Published | Qualified: preliminary report, narrow definition of success | Strategy lead | Jun 2027 |
| C-033 | "Customer data is not used to train the provider's models" | Provider terms of service | Assistant approved for customer emails | Published | Supported; re-check on each terms update | Information officer | Next terms change |
| C-041 | "Credit summaries will double approvals per day" | Internal estimate | Credit-summary build | Asserted | Open: four-week trial running | Credit manager | Dec 2026 |
Three things stand out. First, the claim is recorded in the words used, so nobody can later soften it. Second, the status often splits: C-021 is true for one kind of invoice and false for another, which is exactly the detail that disappears when claims live in slides. Third, every claim expires, because vendor products, models and the organisation's own data change. When the expiry date arrives, the claim is graded again.
A provider-neutral schema
The register does not need special software. A spreadsheet works for a small portfolio; a table in an existing governance or GRC tool works for a large one. What matters is that the fields are consistent. A minimal schema:
claim_id: C-021
wording: "Extracts invoice fields with over 95% accuracy"
claim_type: vendor # vendor | internal | external | risk-regulatory
source: "Vendor sales deck, March 2026"
source_link: null # URL or document reference where available
decisions_supported:
- "Invoice-matching build approval"
evidence_held: "Test on 400 historical invoices, split by format"
evidence_grade: reproduced-here # asserted | published | reproduced-here | observed-in-production
status: qualified # open | supported | qualified | rejected | superseded
status_reason: "Holds for printed invoices; fails on handwritten"
cost_if_wrong: "Clerk rework; supplier payment errors"
owner: "Accounts payable manager"
review_by: 2027-01-31
history:
- date: 2026-08-14
change: "Graded Reproduced here after controlled test"
The cost_if_wrong field deserves attention. It sets how much effort a claim deserves. A claim whose failure would cost little can stay at Asserted. A claim that underpins a large investment or a customer-facing risk should not.
Running it without building a bureaucracy
A register that nobody maintains is worse than none, because it gives false comfort. Four habits keep it alive.
Capture at the decision, not afterwards. Any AI funding, design or risk paper above a set threshold lists its material claims and their grades. The approver checks the list before signing. This takes minutes when the paper is written and hours to reconstruct later.
Set a grade threshold per decision type. Tsitsikamma requires Published or better for a pilot, Reproduced here for scaling, and Observed in production for any figure reported to the board as realised.
Review expiries quarterly. The same quarterly rhythm that reviews the benefits ledger reviews expired claims. Most are re-graded in a few minutes. A few change status, and the decisions they supported are flagged.
Make rejection normal. A rejected claim is a success for the register, not an embarrassment for the person who raised it. The review record for a decision should show which claims were tested and what happened, so that checking becomes routine rather than an accusation.
What the register changes
The register does not make AI decisions for anyone. It makes the reasoning behind them visible and testable. Vendors learn that their claims will be reproduced, and adjust what they say. Teams learn that a survey figure will be graded Asserted, and measure properly. Boards see which figures are evidence and which are hope.
That is the practical meaning of "after the hype". Not a rejection of AI, and not a new wave of enthusiasm, but the ordinary discipline of writing down what you believe, checking it, and changing your decision when the evidence changes.
Next: Part 5, The hype cycle is not an adoption plan. Or see all parts on the series page.
How CloudNala can help
We set up AI claims registers for organisations with growing AI portfolios. We extract the claims behind current AI decisions from business cases, vendor material and board papers, grade the evidence held, design the reproduction tests for the claims that matter most, and connect the register to the benefits ledger and the quarterly review, so it stays small, current and useful.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Value Review or write to us at consult@cloudnala.co.za