AI Strategy20 August 20268 min read

Citizens Build, Agents Execute, Experts Govern — Part 7 of 10

From prototype to production: the trust checklist for AI-built software

Eight questions to ask before a business depends on something that was built quickly. None of them are about whether the code works — that part was never in doubt.

#AI Strategy#Production Readiness#AI Governance#Risk Management

At some point a conversation happens in which someone asks whether a tool that has been working well for a few months can become the official way something gets done. It is usually an informal conversation, it usually happens in a corridor, and the answer is usually yes — because the tool works, and because nobody in the corridor has a framework for saying anything else.

This article is that framework. It is deliberately short and deliberately non-technical, because the people who need to ask these questions are frequently not engineers. What they need is not the ability to answer; it is the ability to ask, and to notice when an answer is missing.

The trust checklist, and when to run itAn AI-built tool that worksIs the business about to depend on this?noAn experiment.Note it, move on.yesTHE TRUST CHECKLISTBUSINESS FITwho owns the outcome?DATAwhat does it touch?SECURITYwho is allowed in?RELIABILITYwhat breaks it?OPERATIONSwho gets the call?COMPLIANCEwhat can we show?COSTwhat does ten times cost?MAINTAINABILITYcan someone elsechange it?A production-readiness decision,made on evidence rather than enthusiasm
Eight questions, none of them technical enough to need an engineer to ask. They need an engineer to answer honestly, which is a different thing — and the honest answer to several of them is often what stops a tool being promoted at all.

First, the branch

The most important step is the one before the checklist: is the business about to depend on this?

If the honest answer is no — one person uses it, and if it broke tomorrow they would go back to doing it by hand for an afternoon — then stop. Note that it exists, and leave it alone. Running eight production-readiness questions against a personal script is how you teach people to stop telling you what they have built.

The checklist earns its cost only when a dependency is forming. That is the moment it is designed for.

The eight questions

Business fit. What problem does this solve, who owns the outcome, and what happens if it produces the wrong answer? The ownership question is the one people skip, and it is the one that determines whether anything else on this list ever gets maintained. "The team owns it" is not an answer. A name is an answer.

Data. What does it read, what does it write, and what does it send anywhere else? For AI tools, add: what goes into the prompt? This catches an enormous number of problems, because prompts frequently contain far more than their author intended — a whole customer record, an entire document, the contents of a spreadsheet — and all of it leaves the building on every single call.

Security. How does it know who is asking? Are credentials stored somewhere other than the code? Can a user reach data they should not see? The most common failure here is not a sophisticated attack; it is a tool built for one person that never acquired a concept of "different users" when other people started using it.

Reliability. What does it depend on, what happens when that thing is unavailable, and what does the user see when it fails? Silent failure is the serious one. A tool that returns nothing when the API is down is annoying. A tool that returns a plausible answer when the API is down is dangerous, and AI-built tools are unusually prone to this.

Operations. Who gets called, how would anyone know it was broken, and where do they look? If the honest answer is that a user will mention it eventually, the tool is not operable, whatever else is true about it.

Compliance. Is there a record of what happened and who approved it? Can that record be produced eighteen months later for someone external? Under POPIA, can you say what personal information is processed, on what basis, and for how long it is kept? We have covered what POPIA means for data and AI in more depth separately.

Cost. What does it cost now, and what does it cost at ten times the volume? Per-call AI pricing means cost scales with usage in a way most internal tools historically did not, and the growth is rarely linear with value.

Maintainability. Can somebody else change this in two years? Is it in version control, is there documentation, is there any test that would catch a mistake? This is the question that decides whether the tool is an asset or a liability with a useful phase.

The checklist is a dial

The single most common way this goes wrong is applying it uniformly.

The checklist is a dial, not a gateREGISTEREDNAMED OWNERACCESS CONTROLAUDIT TRAILFULL GATESPersonal toolTeam workflowDepartment toolEnterprise systemEvery tier is registered. That single column is what turns shadow IT into an inventory,and it costs the builder about thirty seconds.
Applying the bottom row's controls to the top row's tools is the fastest way to teach people to stop telling you what they have built. Proportionality is not a compromise here — it is what keeps the register honest.

Notice that only the first column applies to everything. Registration is nearly free and it is what turns an unknown population of tools into an inventory. Everything else is earned by consequence.

Applying the bottom row's controls to the top row's tools is not caution — it is the fastest way to make the register dishonest. If registering a personal script triggers an architecture review, people stop registering personal scripts, and you lose the visibility that made the whole thing work.

How to actually run it

An hour, four people, one document. The person who built it, someone who understands the business process, someone who can answer security and data questions, and someone with the authority to decide.

Go through the eight questions and write down the answer. Where there isn't one, write "unknown" rather than guessing — the unknowns are the output. Then make one of three decisions: promote it with the gaps closed first, keep it as a team tool with its current limits made explicit, or replace it because the requirement is now clear enough to build properly.

That third outcome is common and should not be treated as a failure. The prototype did its job: it established that the problem was real and specified the solution in unusual detail. Rebuilding it on an approved path is frequently cheaper than remediating it, especially now that rebuilding is fast.

What honest answers look like

A short calibration, because "we have logging" can mean several things.

Weak: "It's secure, only our team has the link." Strong: "It requires a company login, and it checks the requester's department before returning anything."

Weak: "It's been fine." Strong: "It has failed twice, both times because the source system was down, and the user sees an error rather than an empty result."

Weak: "The data is just internal." Strong: "It reads three named fields from the CRM, sends none of them to the model provider, and stores nothing."

Weak: "I support it." Strong: "I support it, my backup is named, and the runbook is in the repository."

The pattern is that strong answers are specific and checkable. If every answer in the room is a reassurance rather than a fact, the tool is not ready, and the room now knows it.

How CloudNala can help

Running this review is a large part of what we do — usually on a tool that already works and already matters, where the organisation needs an independent read on what depending on it would actually require. The output is a short list of gaps ranked by consequence, a recommendation on promote, contain or rebuild, and where a rebuild is right, a design that reaches the same outcome on a supportable footing. The related article on taking an AI prototype to production covers the agent-specific version of this in more depth.


Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Readiness Workshop or write to us at consult@cloudnala.co.za