The earlier parts of this series used AI test tools as helpers: drafting cases from acceptance criteria, back-filling traceability links, clustering failures, summarising results. All of that is valuable. It also understates what a modern AI test agent actually does. It executes code. It calls APIs. It creates and deletes test data. It reads logs. It can edit test files and, if allowed, mark tests as skipped or change their expected results.
An agent with those abilities is not a helper sitting beside the process. It is a privileged actor inside the process, with access to environments, data and the evidence record itself. It needs guardrails designed for that, in the same way a new team member with production access would. This final part sets them out.
Kudu Pharmacies is fictional, and its systems and figures are illustrative. Kudu's AI test agent works on the Kudu Repeat app. Because the app handles prescriptions, the data it touches includes health information about patients.
Four guardrail areas
The guardrails fall into four areas: where the agent may act, what data it may use, what it may change in the evidence, and what needs a person's approval.
Environment boundaries
The agent works in test environments only. That sounds obvious, and it is surprisingly easy to get wrong. Test environments often hold credentials for shared services, connections to third-party sandboxes that are not always sandboxed, or configuration that points at production for convenience. An agent that follows a misconfigured connection string can reach production without anyone intending it.
Practical controls:
- A scoped identity for the agent, separate from any human's, with permissions only in test environments.
- Network rules that make production unreachable from where the agent runs, rather than relying on the agent not to try.
- Destructive actions limited to data the agent created. The agent may delete its own test records; it may not truncate shared tables.
- Third-party calls through stubs or verified sandboxes, never live partner endpoints. A test agent that submits a real medical aid claim is an incident, not a test.
These are the same least-privilege ideas covered in Least-privilege as code, applied to a non-human actor that moves quickly and does not pause to wonder.
Test data
Test data is where AI test automation meets privacy law. Kudu's data includes prescriptions, which are health information. Under POPIA, health information is special personal information, and processing it starts from a prohibition with specific exceptions, as our AI and POPIA series explains in plain language. This is general information, not legal advice; the determination belongs with your privacy and legal people.
The practical answer is tiers:
- Synthetic data is the default. Generated records with realistic shapes and no real patients behind them. AI is good at generating these, which is one of its most useful contributions to testing. The agent may use synthetic data in any test environment.
- Masked copies are the exception. Where realistic distributions matter, for example performance testing at real volume, a masked copy may be approved for a restricted environment with a retention limit. The approval sits with a privacy owner, not the test team.
- Production data is not available to test agents. Not read access, not "just for debugging", not through logs that contain unmasked payloads.
The last point catches teams out. Application logs in test environments sometimes contain production data copied in for debugging months ago. An agent reading those logs to diagnose a failure is now processing personal information nobody approved. Log hygiene in test environments is part of the guardrail.
Evidence protection
The agent must not be able to weaken the evidence that release decisions rely on. In practice, it may not:
- delete or overwrite test results, which the append-only store from Part 6 enforces technically;
- mark tests as skipped or quarantined on high-risk criteria without approval;
- change expected results on tests that cite high-risk acceptance criteria without review;
- edit the release record, which is generated from the evidence, not authored.
The agent may propose all of these changes. Proposals go through review like any other change. This matters most when the same family of model writes the code and maintains the tests, because the easiest way for an agent to resolve a disagreement between code and test is to change the test. The Coding Agent Harness series covers the checks around coding agents in general; test automation is where this particular risk bites hardest.
Reviewer approval
Some actions always need a person: changing tests on high-risk criteria, quarantining a test from release gates, approving use of masked data, and anything that affects the release decision. The reviewer should be someone other than the person who configured or prompted the agent for that task, so the review is a genuine second look.
Logging what the agent did
Every agent action should leave a record: what it was asked to do, what it did, which environment and data it touched, and what changed as a result. This is partly for security and partly for the release record itself. Part 2 argued that the release record should distinguish agent work from human decisions. The agent's own log is where that distinction comes from.
At Kudu, the record for release 4.12 shows that the agent generated 41 of the tests cited for high-risk criteria, proposed two test changes that were reviewed and accepted, proposed one quarantine that was rejected, and used synthetic data only. That is a short paragraph, generated automatically, and it answers most of the questions an auditor or an incident review would ask about the agent's role.
Bringing the series together
Across nine parts, Release on Evidence has argued one thing from several directions: a release decision is only as good as the chain of evidence behind it, and AI makes that chain both easier to build and easier to fake.
- Part 1 set out the chain, from intent to decision.
- Part 2 separated AI preparation from human accountability.
- Part 3 gave the minimum metadata to hold the chain together.
- Part 4 widened readiness beyond functional tests.
- Part 5 replaced activity reporting with evidence quality.
- Part 6 kept failures instead of hiding them.
- Part 7 made evidence survive intermittent connectivity.
- Part 8 kept scores explainable.
- This part put guardrails around the agent itself.
None of these steps requires a new platform. Each is a set of conventions, checks and records that a team can introduce into the tools it already has, starting with its five riskiest requirements. All parts are on the series page. For the wider delivery picture these practices sit inside, see The AI-Native Delivery Loop.
How CloudNala can help
We help teams introduce AI test automation safely: defining the agent's identity and environment boundaries, setting up test data tiers with the right approvals, protecting the evidence store and designing the review points that keep accountability with people. Where an agent is already in use, we review what it can reach and change, and close the gaps.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book a Release Evidence Review or write to us at consult@cloudnala.co.za