When Ukhozi's privacy officer asked a simple question, "where do customer prompts end up?", the platform team expected a short answer. Prompts go to the provider, the provider has a contract, done.
The real answer took a week to assemble. A single customer-service prompt, in which a customer had typed their ID number to prove who they were, existed in at least five places: the gateway's request log, the tracing tool the developers used to debug the agent, the provider's abuse-monitoring store, the security team's log export, and the nightly backups of the gateway's database. Each copy had a different retention period, a different access list, and in two cases no deletion process at all.
Nobody had decided to keep five copies. Each was the default of a tool doing its job. CloudNala's position: observability is a data-handling decision. Decide per field what is stored, redacted or never kept, apply the same rules to every copy, and test deletion.
Ukhozi Telecom is fictional, and its systems and figures are illustrative. This part is general information, not legal advice. Your privacy and legal advisers should make the determinations for your organisation.
Where the copies come from
The gateway request log. Many gateways can log full request and response bodies. It is invaluable during the first proof and often left on afterwards.
Tracing tools. LLM tracing products record each step of an agent run, including prompts, retrieved context, tool inputs and outputs. That is exactly what makes them useful for debugging, and exactly why they become a second store of personal information. The Agent Evidence Layer explains why traces matter; this part is about what they hold.
Provider-side retention. Providers typically retain inputs and outputs for a period for abuse monitoring unless an exception has been agreed. The terms differ by provider, product and contract, and change over time, so check the current terms for your actual agreement rather than assuming.
Security exports. Gateway logs are often shipped to a SIEM for security monitoring. If the log line contains the prompt, so does the SIEM, usually with a long retention period and a wide audience.
Backups. Whatever is in the gateway's database is in its backups. Deleting a record from the live system does not delete it from last month's backup.
Each arrow in the diagram is a separate retention decision. Under POPIA the organisation remains the responsible party for all of them, and the condition that personal information be kept no longer than necessary for the purpose applies to the debug copy as much as to the customer record. What POPIA means for your data and AI strategy covers the broader principles.
Decide per field, not per log
The fix is not to turn logging off. Without telemetry, the gateway cannot do the jobs that justified it: attribution, cost control, incident investigation. The fix is to separate metadata, which is needed and low risk, from content, which is useful occasionally and high risk.
Metadata (caller, key, model, route, token counts, cost, latency, status, policy decisions) is what operations and finance need every day. It rarely contains personal information, and Ukhozi keeps it for thirteen months so year-on-year comparison is possible.
Content (prompt and response text) is what developers want when something goes wrong. Ukhozi's rule: content is redacted before storage, and only a small sample is kept, for thirty days. When an incident needs full content, the application owner enables full capture for that route for a limited window, with the change recorded.
Detected personal information is never stored. Redaction runs before anything is written, replacing ID numbers, phone numbers, account numbers and email addresses with typed placeholders. The detection was tested with realistic formats, including the spaced ID numbers that defeated the first version in Part 2.
Provider error bodies are a surprising leak: some error messages echo part of the request. They are redacted like content.
Apply the rules to every copy
A policy for the gateway log is not enough if the tracing tool and the SIEM keep their own copies. Ukhozi's review produced one table per destination, with the same field rules:
| Destination | Content stored? | Retention | Who can read it | Deletion tested? |
|---|---|---|---|---|
| Gateway log | Redacted sample | 30 days | Platform team | Yes |
| Tracing tool | Redacted, production; full, development only with synthetic data | 14 days | Application team | Yes |
| SIEM | Metadata only | 12 months | Security team | Yes |
| Provider | Per contract; exception requested for zero retention on two routes | Per contract | Provider | Contractual |
| Backups | Follows source, with backup expiry aligned | 35 days | Platform team | Yes |
The tracing tool was the hardest change, because developers valued full production traces. The compromise was synthetic test data in development, redacted traces in production, and a time-limited, logged exception for live incidents.
Access and deletion are part of the design
Two further questions belong in the architecture, not in a policy document:
Who can read stored content? Access to prompt logs is access to whatever users typed. It should be as restricted as access to the underlying customer records, and reviewed in the same cycle.
Can a specific record be found and deleted? If a customer asks what Ukhozi holds about them, or a prompt is found to contain something that must be removed, the team must be able to find it in every store and delete it. Ukhozi tested this by planting a marked synthetic prompt and confirming it could be located and removed everywhere, including the expiry of the backup that contained it.
The short version
Count the copies. One prompt can live in five places by default.
Metadata and content are different. Keep the first, minimise the second.
Redact before storage. And test redaction with real-world formats.
Same rules for every destination. Including traces, SIEM and backups.
Prove deletion. A retention period nobody can enforce is a wish.
Previous: Part 5, failover can improve availability and damage consistency. Next: Part 7, a working admin screen is not an operated platform. All parts are on the series page.
How CloudNala can help
We map where AI prompts and responses are stored across gateways, tracing tools, security exports, providers and backups, then help you set field-level rules for storing, redacting and deleting them, with access and deletion tested rather than assumed. Your privacy and legal advisers stay responsible for the determinations; we give them an accurate picture to decide on.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Platform Architecture Review or write to us at consult@cloudnala.co.za