The demo is the beginning

A prototype shows that an agent can complete one task when the data is clean, the tools are available and the user asks the expected question. That is worth knowing. It is also very little.

Production needs evidence across the conditions that make the work difficult, and a forward deployed team has to produce four kinds of proof, not one:

1. Technical behaviour: the system does what it was built to do, including when things fail.
2. Task quality: its outputs are good enough for the people who rely on them.
3. Operational reliability: it can run, be supported and recover.
4. Business outcome: the process actually improved.

Moza's three-stage model puts evaluation in the middle for exactly this reason: evaluations are how unpredictable AI behaviour becomes evidence someone can make a decision on. This part is about building that evidence.

From demonstration to controlled production

Sandveld Foods and Ironbark Digital are fictional, and their systems and figures are illustrative.

Sandveld's first AI invoice product stopped at the demo (Part 1). Ironbark's pod took the invoice workflow through six stages, each proving something the previous one could not.

Prototype. Prove the core interaction is technically possible, using safe or synthetic data. Ironbark's engineer built the first extraction and explanation flow on synthetic invoices modelled on the variants from Part 2.

Hardened workflow. Add schemas, authentication, permissions, error handling, idempotency, logging, rate limits, budgets and recovery. This is the unglamorous stage that most demos skip, and it usually takes longer than the prototype.

Evaluation suite. Measure representative normal, edge, failure and adversarial cases, and define what blocks release and what routes to human review.

Shadow operation. Run alongside the existing process without making decisions, and compare outputs, errors, cost and timing. Sandveld's shadow ran for four weeks beside the clerks, who kept working exactly as before.

Canary release. Use a controlled portion of users or cases, with rollback and escalation available. Sandveld started with the suppliers to one plant, Wellington, before extending to the others.

Bounded production. Operate within approved users, actions, budgets and data, and monitor technical and business measures. Autonomy increases after this, step by step, and only when the evidence supports it.

Build for the unhappy paths

Moza places a high value on exception handling, and rightly. A production test pack should include:

• missing and contradictory inputs;
• damaged or unreadable documents;
• duplicate requests;
• stale knowledge;
• tool timeouts and provider outages;
• permission and authentication failures;
• rate limiting;
• prompt injection in retrieved content;
• the wrong customer or supplier context;
• human rejection and correction;
• workflow cancellation;
• a retry after a side effect;
• a cost or iteration limit being reached.

The test does not only ask whether the system failed. It asks whether it failed safely, said so, and left enough evidence to recover.

Shadow operation earned its place at Sandveld in week two. A grower sent an invoice as a phone photo and, a few days later, sent the same invoice as a PDF, with the invoice number formatted differently. The agent treated them as two invoices. The senior clerk caught it, as they had been catching them for years. The pod added a duplicate check that compares supplier, amount, date and delivery rather than relying on the invoice number, and added the case to the golden set so it can never silently regress.

The golden set also contains a planted request to change a supplier's bank details, written to look like a routine supplier letter. The pass condition is that the agent does nothing with it except stop and route it to the verification process from Part 3.

What the shadow run measured

Shadow operation is only useful if the comparison is defined before it starts. Sandveld's pod agreed four comparisons with the AP manager in advance:

• Agreement. For each invoice, did the agent reach the same outcome as the clerk: post, request a delivery note, flag a variance, block as a duplicate? Disagreements were reviewed one by one, and each was classified as an agent error, a clerk error or a genuine judgement call.
• Time. How long did the clerk spend on the same invoice, and how much of that time would the agent's draft have saved?
• Cost. What did each invoice cost to process, including retries, against the budget the CFO had approved?
• Safety. Did the agent ever propose an action it was not permitted to take?

The disagreements were the most valuable output. Several were clerk errors, which the AP manager used for coaching rather than blame. Most of the rest were cases the golden set had not covered, which went straight into it. The canary did not start until a full week of shadow produced no new kind of disagreement.

The evaluation portfolio

A worked scorecard: supplier invoice exceptions

At Sandveld, the agent extracts invoice data, checks it against the purchase order and delivery, and drafts an exception recommendation. The release scorecard the AP manager signed off:

The figures are illustrative. Real thresholds should follow the value and consequence of the workflow. "Representative documents" matters in the second row: Sandveld's set had to include phone photos in the same proportion as real traffic, because extraction accuracy on clean PDFs says almost nothing about a photo of a grower's handwritten invoice taken in a packshed.

Each threshold is tied to a response decided in advance. A latency breach triggers an alert; a missed duplicate blocks release. Evals that gate real work covers how to connect each result to an action and an owner.

Observability is part of the deliverable

Every material run should connect:

• the user, tenant and task;
• agent, prompt, workflow and policy versions;
• model and tool calls;
• memory and evidence retrieved;
• cost and latency;
• errors, retries and fallbacks;
• the evaluation result;
• human approval or correction;
• the final system update;
• the business outcome.

This is the agent evidence layer described in our Agent Evidence Layer series. Without it, a forward deployed team cannot explain performance, investigate incidents or prove value, and the client cannot operate the service after the team leaves. At Sandveld, every invoice carries one trace from the mailbox to the ERP posting, which is how the pod could show, during shadow, exactly where the duplicate slipped through.

Production exit criteria

Before bounded production, the AP manager and Sandveld's Head of IT required:

• Named business and service owners.
• A representative evaluation set and approved thresholds.
• A completed security and privacy review.
• Data, identity and regional constraints tested.
• Error, retry and recovery behaviour proven.
• Cost budgets and alerts active.
• Human escalation staffed and measured.
• A support runbook and rollback prepared.
• Adoption and training complete for the first users.
• Business baseline and outcome measurement agreed.

The short version

A demo proves possibility, not production. Technical behaviour, task quality, reliability and outcome all need their own evidence.

Move through stages. Prototype, harden, evaluate, shadow, canary, bounded production.

Test the unhappy paths. Ask whether it failed safely, said so and left evidence.

Let shadow find what tests missed. Then turn every finding into a golden-set case.

Ship the evidence with the software. Observability is part of the deliverable.

The FDE's work is complete when the client can operate the service safely, not when the demonstration earns applause.

Previous: Part 3, where should intelligence live? Next: Part 5, build into the business you actually have. See every part on the series page.

How CloudNala can help

We take AI workflows from demo to bounded production: hardening the prototype, building evaluation suites from your own cases including the unhappy paths, running shadow and canary stages against the existing process, and agreeing exit criteria with your business and IT owners. The evidence layer is part of what we hand over, so your team can operate and improve the service after we leave.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Opportunity Sprint or write to us at consult@cloudnala.co.za