Release on Evidence
AI can now write tests faster than any team can read them, and a green suite proves less than it used to. This nine-part series sets out how to turn testing into evidence a release decision can rely on: a traceable chain from requirement to decision, a clear line between AI preparation and human accountability, the minimum metadata, readiness beyond functional tests, quality reporting that shows evidence rather than activity, failed results that are kept, assurance where connectivity is intermittent, explainable readiness scores and guardrails for AI test agents. One fictional South African pharmacy chain and its repeat-prescription app carry the examples, and every figure is illustrative.
9 articles
- Part 1A test without a requirement is orphan evidenceAI can generate thousands of passing tests in an afternoon. None of them help a release decision unless each one can be traced to a requirement, a build and a named decision. Part 1 of Release on Evidence sets out the evidence chain.8 min read
- Part 2AI can draft the test. It cannot own the releaseAI test tools can write cases, generate data, run suites and summarise failures. None of that transfers accountability for putting a build in front of customers. Part 2 draws the line between AI assistance and the release decision.6 min read
- Part 3From work item to release decision: the minimum traceability metadataTraceability fails when it is treated as a documentation project. It works when a handful of identifiers travel automatically between the systems a team already uses. Part 3 sets out the minimum metadata and how to thread it.6 min read
- Part 4A passed test is not proof of production readinessFunctional tests prove the build does what its criteria say. Production asks six more questions: security, performance, resilience, operability, data and rollback. Part 4 widens the release gate without turning it into bureaucracy.6 min read
- Part 5Quality intelligence is more than a test dashboardMost test dashboards report how busy the testing was: tests run, pass rate, coverage percentage. A release owner needs to know how much the evidence can be trusted and where it is thin. Part 5 separates activity from evidence quality.6 min read
- Part 6Why failed tests must keep their evidenceWhen a rerun turns red to green and the first result is overwritten, the team loses the one record that might have explained a production incident. Part 6 makes the case for append-only test evidence and shows what a failed result should keep.6 min read
- Part 7Offline-first assurance for intermittent environmentsRelease evidence usually assumes a test environment that is always connected. Stores, clinics, depots and branch offices are not. Part 7 shows how to capture trustworthy evidence where connectivity comes and goes, without treating silence as success.6 min read
- Part 8Release readiness scores need an explanationA single readiness number is attractive to busy approvers and dangerous for the same reason. Part 8 shows how to keep a score useful: visible inputs and weights, recorded exclusions, named overrides and hard gates the number cannot outvote.6 min read
- Part 9AI test automation needs its own guardrailsAn AI test agent runs code, touches environments, handles data and edits the evidence that release decisions rely on. That makes it a privileged actor, not a helper. Part 9 sets out the guardrails: environment boundaries, data tiers, evidence protection and reviewer approval.6 min read
Working through any of this in your own organisation?
These articles come out of delivery work rather than theory. If the problems in this series look familiar, a short scoping conversation is usually the quickest way to work out what is worth doing first.
Talk to CloudNala