Most organisations that use AI to help with decisions have an appeal process on paper. A person who disagrees can write in, someone senior looks at the case again, and the outcome is confirmed or changed. The process exists. Whether the systems underneath can support it is another question.
The elements of a fair challenge process (notice, explanation, a chance to respond, review by someone with power to change the outcome) are set out in A person needs a route to challenge the outcome, which also covers what section 71 of POPIA does and does not require. This part assumes that process exists and asks the engineering question: what must the decision system be able to do so that an appeal is a genuine reassessment rather than a second person glancing at the first person's screen?
At Amatola Bursary Agency, the fictional provincial agency in this series, the first appeals after the AI pilot exposed four gaps. The appeal reviewer could only see the current rule, not the one that applied. Correcting a wrong income figure fixed it in one system and not in two others. The original evidence had been overwritten by the corrected version. And when an appeal showed that a rule had been coded wrongly, nobody could find the other applicants it had affected. Each gap is a design problem with a design answer.
The appeal path as a system
The path itself is simple: decision and reasons, appeal lodged, reassessment, outcome, correction. The architecture is in what each step needs from the systems underneath.
Reassess under the original rule
An appeal asks whether the original decision was right. That means reassessing under the rule that applied when the original decision was made, not the rule in force today. If the policy changed in the meantime, the appellant does not automatically benefit from the change or suffer from it, unless an approved transitional provision says so.
This is where the date-aware rules register from Part 2 earns its keep. Amatola's rules engine accepts an "as at" date. The appeal reviewer runs the case as at the original application date, with any corrected facts, and sees the rule trace for that version. The case file from Part 4 supplies the original inputs, so the reviewer starts from exactly what the first decision saw.
The reassessment also has to be independent. Amatola assigns appeals to a reviewer who did not make or approve the original decision, and the assistant's original recommendation is shown as part of the record, clearly labelled, rather than as a suggested answer. Presenting the first recommendation as the default invites the reviewer to confirm it.
Separate three kinds of error
Appeals succeed for different reasons, and the system should record which one.
| Error type | Example (illustrative) | What has to change |
|---|---|---|
| Fact error | Household income was extracted from the wrong page of a payslip | This case's facts, everywhere they were copied |
| Application error | The right rule, wrongly applied: an exception that should have fired did not | The engine or decision table, then every case it affected |
| Rule error | The rule itself, or its interpretation, produces an outcome the policy did not intend | The structured policy, through approval, then every affected case |
The distinction matters because the remedy is different. A fact error is fixed for one person. An application or rule error is a defect that has probably affected others.
Correct the facts everywhere
Decision systems copy facts. The income figure extracted from a payslip goes into the case management system, the rules engine input, the payment system's eligibility flag and a reporting extract. Correcting it in the appeal screen and nowhere else leaves the error alive in three other places, one of which may stop a payment next term.
Amatola treats a corrected fact as an event. The appeal records the correction once, with the evidence and the reviewer, and the correction is propagated to every system that holds a copy, with each update logged against the case. This mirrors the correction obligation in section 24 of POPIA and is the same propagation problem described in An access request is a test of your whole system. In both cases, the hard part is knowing every copy exists.
Preserve the original
The temptation, when correcting a case, is to overwrite the wrong data with the right data. That destroys the evidence of what the original decision was based on. If the appellant later complains about the original decision, or an auditor asks why it was wrong, the agency can no longer show what happened.
The case file is append-only. Corrections are added as new entries that supersede earlier ones, with the original preserved. The reasons letter for the appeal outcome cites both what was originally found and what was corrected. This is not just good record-keeping; it is what lets the agency demonstrate that an appeal process actually changes outcomes.
Find everyone else the error touched
The most important appeal outcome is often not for the appellant. When an appeal shows that a rule was coded wrongly, or that an interpretation does not match what the policy intended, every other case decided under that rule may be wrong too. Most of those applicants will never appeal. Some will not know they could.
Because each case file records the rule versions applied, Amatola can query for every case decided under the faulty version, rerun them under the corrected rule in a shadow mode and produce a list of cases whose outcome would change. The agency then decides, with its policy owner and legal advisers, how to remedy those cases. The appeal by one person becomes a correction for everyone affected, which is what a fair system should do.
Without per-case rule versions, this query is impossible, and the agency is left choosing between ignoring the problem and reviewing every case by hand.
Appeals as a signal about the rules
Amatola counts successful appeals by rule, by interpretation record and by error type. A rule with a cluster of successful appeals is a rule to review, even if each individual appeal looks like a one-off. The rule owner receives that count every month as part of the appeal pattern review shown in the diagram.
This closes the loop the series has been building. Rules are made explicit (Part 1), made authoritative (Part 2), checked for conflicts (Part 3) and applied with a full case file (Part 4). Appeals test them against real lives, and the results feed back into the rules through the same controlled change process covered in Part 8.
For public bodies, the requirement to give adequate reasons on request under the Promotion of Administrative Justice Act sits alongside internal appeal and judicial review routes, and Human-in-the-loop is not administrative justice considers those obligations in more depth. This article is general information, not legal advice.
Questions to test your appeal path
- Can a reviewer run a case under the rule version that applied on the original date?
- Does a corrected fact reach every system that holds a copy, and can you prove it did?
- Is the original evidence preserved after a correction?
- Can you list every case decided under a specific rule version within an hour?
- Who receives the count of successful appeals by rule, and what do they do with it?
Next: Part 6, The model should not be the law. All parts are on the series page.
How CloudNala can help
CloudNala helps teams make their appeal path work at the system level. We design the "as at" rule evaluation, the correction event that propagates to every copy, the append-only case file and the query that finds every case a faulty rule touched, and we set up the appeal pattern reporting that turns individual appeals into rule improvements. We work alongside your legal advisers and do not give legal advice.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Decision Review or write to us at consult@cloudnala.co.za