Most organisations collect a great deal about how their systems behave in production. Dashboards show latency and errors. Support tools record tickets. Product analytics show where users drop out. Finance sees the bill. Incident reviews produce actions.
Very little of it reaches the people deciding what to build next in a form they can act on, and almost none of it reaches the agents. Agents build from their context: the intent, the specification, the architecture rules, the harness. If production teaches you that a rule was wrong, but the specification still contains the old rule, the next agent task will rebuild the mistake with complete confidence.
That is why we say production is part of the prompt. Not literally, as in pasting log lines into a chat, but structurally: production behaviour has to change the artefacts agents read, or the AI-native loop from Part 1 is not a loop at all. It is a pipeline that ends at release.
Seapoint Payments is fictional, and its systems and figures are illustrative. Seapoint's onboarding squad shipped automated bank-account verification and the partial-match payout hold in the previous parts. This part follows what production told them in the first month.
Three signals from Seapoint's first month
Seapoint's onboarding journey went live to all new merchants. Within four weeks three signals stood out.
- Abandonment at document upload. Around 18% of merchants who started onboarding on a phone left at the step where they upload company registration documents. On a laptop the figure was much lower.
- Support tickets about held payouts. Merchants with partial matches, mostly sole proprietors trading under a different name from the one on their bank account, were opening tickets to ask why their money was held.
- Verification timeouts at peak. At month-end, when onboarding volume spikes, the external verification service slowed, and some checks timed out.
Each of these is the kind of thing that, in many organisations, becomes a dashboard tile or a ticket in someone's backlog. At Seapoint, each became a change to a specific artefact.
Route the signal to the artefact that caused it
The question for each signal is not "who should fix this?" but "which artefact produced this behaviour?" That artefact is what must change, because it is what the next cycle, and the next agent, will build from.
Abandonment is an intent problem. Seapoint's success measure was "time to first payout". It said nothing about completion rate on mobile. The product owner revised the intent page to add a completion measure by device and a constraint that every onboarding step must work on a mid-range phone over a mobile connection. That constraint then flowed into the specification and the acceptance criteria. Without it, the next agent task would have optimised payout time again and ignored the drop-off entirely.
The held-payout tickets were a specification problem. The rule from Part 2, hold on partial match, was right for fraud control and wrong in its treatment of sole proprietors trading under a name, which is common and legitimate. The analyst and risk lead added a rule: for sole proprietors, a partial match where the account holder's name matches the registered owner's name is treated as a match. A new acceptance criterion and examples followed. The merchant notification was also rewritten to explain what the merchant could do.
The timeouts were an architecture and harness problem. The architect added a limit to the specification (verification may be queued and retried for up to fifteen minutes at peak, with the merchant told it is in progress) and a sensor to the harness: a load test against a simulated slow verification service, run before any change to the verification path merges. The next agent to touch that code met the sensor before production did.
The weekly triage
None of this happens without a routine. Seapoint runs a short weekly triage with the product owner, analyst, architect, tester and operations lead. The agenda has three questions.
- What did production show us this week? Funnel data, tickets, incidents, latency, cost and any manual interventions by operators.
- Which artefact produced each behaviour? Intent, specification, architecture, harness, or runbook.
- Who changes it, by when? The owner of that section, under the same approval rules as any other change.
The same session reviews the harness backlog from Part 4, because repeated defects in production are often missing sensors. It usually takes half an hour.
Signals worth routing, by type
| Signal | Where it usually comes from | Artefact it most often changes |
|---|---|---|
| Users abandon or work around a step | Product analytics, session data, support | Intent and success measures |
| A rule gives the wrong outcome | Tickets, complaints, risk reviews | Specification and acceptance tests |
| The same defect class recurs | Incident reviews, escaped-defect logs | Harness: a new guide or sensor |
| Latency, cost or capacity drift | Monitoring, cost reports | Architecture and non-functional limits |
| Operators fix things by hand | Operations logs, runbook use | Operational section and automation backlog |
| Agent-built changes cause rework | Review and acceptance records | Guides, templates and specification clarity |
The last row closes a loop that is easy to miss. The delivery process itself produces signals. If agent-built changes keep failing acceptance for the same reason, that is evidence about the specification or the harness, and it should be routed just like a production incident.
What to be careful about
Do not paste raw production data into agent context. Logs and tickets contain personal information. Seapoint's merchants are businesses, but their directors and sole proprietors are people, and POPIA applies. Route the signal, in summarised and minimised form, to the artefact. Never route customer records into an agent session.
Do not route every signal. Noise overwhelms a triage quickly. Start with the funnel, tickets and incidents for one capability, and add sources only when the team can act on them.
Do not skip the owner. A production signal is not permission for an agent to edit the specification. Agents can propose changes, and can be very useful at summarising signals into a draft, but the section owner decides.
The broader point
Teams often describe AI-native delivery as faster build cycles. The more important change is a shorter distance between what production teaches and what the next cycle builds. When agents can turn a corrected specification into working code in hours, the bottleneck becomes how quickly the organisation learns and records the correction. Weekly triage, artefacts with owners and a habit of asking "which artefact caused this?" are what make that distance short.
For the operational side of this, our Agent Evidence Layer series covers traces, evaluations and operating models for agents in production, and Production begins after deployment covers running an enterprise AI product once it is live.
Next: Part 8, AI-native does not mean AI everywhere. All parts are on the series page.
How CloudNala can help
We help teams close the gap between production and the next build cycle: choosing which signals to route for one capability, setting up a short weekly triage, mapping each signal type to the artefact and owner that should change, and making sure personal information stays out of agent context while the lessons still get through.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Plan an AI Delivery Pilot or write to us at consult@cloudnala.co.za