A customer opens a chat and types: "My dining table was meant to arrive on Monday. Order SB-2026-118204." The assistant checks the order, contacts the courier and replies sensibly. The customer asks: "So when will it actually arrive?" The assistant answers: "Could you give me your order number?"
Nothing has crashed. Every component did exactly what it was built to do. The customer, though, has just learnt that the assistant is not listening, and she will probably ask for a person.
This is the most common first complaint about AI assistants, and it is usually described as a memory problem. That framing pushes teams straight towards vector databases, memory products and long-term customer profiles. Most of the time the actual fix is much smaller and much less risky. This seven-part series is about the difference: what an agent should remember, where it should keep it, for how long and under whose authority. Getting those decisions right matters more than which memory product you buy.
The shop that brought us here
Sunbird Home is fictional, and its systems and figures are illustrative. Sunbird is an online homeware retailer in Cape Town. It ships about 40,000 orders a month through two courier partners, and its support team of 18 people handles questions in English, Afrikaans and isiXhosa. Last year it added a support assistant, Sunbird Help, to its website chat and its WhatsApp line. The assistant handles about 11,000 conversations a month, and roughly four in ten are about orders that are late, split or missing.
Our running customer is Zanele Mokoena, who lives in Gqeberha and ordered a dining table. Her order is SB-2026-118204. The courier has opened query CQ-55190 because the table missed its delivery slot. Zanele is the customer in the exchange above.
Sunbird's first version of the assistant handled single questions well. It struggled the moment a conversation needed more than one turn.
Every model call starts from nothing
A large language model does not keep a conversation in its head between messages. Each request to the model is independent. It receives some text (instructions, the conversation so far, any retrieved information) and returns a response. When the next message arrives, the model has no record of the previous call unless the application sends that record again.
The "chat" a customer experiences is something the application builds. Behind each reply, software decides what to include in the next request. If the application sends only the latest message, the model sees "So when will it actually arrive?" with no order number anywhere in sight. Asking for it is the correct response to what the model was given.
That is why the problem is an architecture decision rather than a model defect. A more capable model would ask the same question, because it would be given the same incomplete input. What changes the behaviour is what the application chooses to carry from one call to the next.
What the customer actually experiences
The repeated question is the visible symptom. There are three quieter ones that cause more damage.
Contradiction. If the assistant re-derives everything every turn, it can give a different answer to the same question two messages apart. In one turn it says the courier expects Friday, and in the next it says delivery is "within 5 to 7 working days" because it has fallen back to the generic policy text.
Repeated verification. Sunbird verifies a customer before discussing an order by sending a one-time PIN to the phone number on file. Without state, the assistant cannot tell whether verification has already happened, so it either asks again, which is irritating, or assumes it has, which is a security problem.
Lost progress. The assistant opened a courier query in turn one. Without a record of that, a later turn can open a second query for the same parcel, and the courier now has two tickets for one table.
Each of these is a failure of continuity within a single conversation. None of them needs the assistant to remember anything about Zanele beyond the next few minutes.
The first fix is state, not memory
The smallest thing that solves all four symptoms is working state: a small, structured record the application keeps for the duration of a task. It holds the facts the task depends on, and nothing else.
Three properties make this work.
It is typed. Each field has a name and an expected form. order_id is an order identifier, confirmed against the order system, not a phrase the model thought looked like one. That makes it safe for software to act on.
It is written by the application, not by the model's judgement. The workflow sets step when it moves the conversation on. The verification service sets customer_verified. The courier API returns courier_query. The model can propose that the customer has mentioned an order number. The application confirms it before it becomes state.
It expires. Sunbird deletes working state 48 hours after the last message in a conversation. It is not archived and it does not quietly become part of a customer profile. If Zanele comes back next week, the assistant starts again, confirms her identity and looks up the order, which is what she would expect from a careful person.
With that record in place, the second turn arrives with the order number, the verification flag and the open courier query. The assistant answers "the courier expects to deliver on Friday" without asking anything, does not open a duplicate query and does not re-verify. Most teams that believe they have a memory problem have this problem, and it can be fixed in days.
Where state lives, and where it does not
Working state belongs in the application's session layer: a key-value store or a database table keyed to the conversation, with a time-to-live. It is ordinary engineering. You already know how to secure it, back it up (or decide not to), and delete it.
What it should not be is the model's context window treated as storage. Some teams solve the repeated question by resending the whole transcript on every turn and hoping the model finds the order number in it. That works in a demo and becomes expensive, slow and risky in production, for reasons Part 2 sets out with numbers.
Nor should working state be a long-term memory store. The moment the order number, the verification status and the courier query start persisting beyond the conversation, Sunbird is holding personal information for longer than the purpose needs, and it now has a store that has to be governed, corrected, secured and eventually deleted. That may be justified for some information. It is not justified for this.
When state is not enough
Working state answers "what is this conversation about?" It cannot answer three other questions that come up quickly in customer service:
- What happened before? Zanele contacted Sunbird twice last month about a different order, and the second time a supervisor promised a delivery-fee refund that has not been paid. That is episodic information.
- What do we know about her that is still true? She asked to receive updates on WhatsApp rather than by email. That is a stated preference.
- How should this kind of case be handled? Sunbird's delayed-order procedure says when a customer may cancel for a full refund. That is procedure, and it changes over time.
Each of these is a different kind of memory with different rules about who may write it, how long it lasts and how it is corrected. CloudNala has covered the broad picture elsewhere, in memory for AI agents, memory as a designed layer and memory as a governed data product. This series goes further into the design decisions, using one retailer and one customer throughout.
What this series covers
- Part 2 explains why sending the whole conversation history is the most expensive form of memory, and what to assemble instead.
- Part 3 separates working state, episodic, semantic and procedural memory, and shows why procedural memory is really versioned business policy.
- Part 4 argues that the write policy (what is allowed into memory) matters more than the database.
- Part 5 shows why retrieving a memory is not the same as being entitled to use it.
- Part 6 turns POPIA into design questions for a memory store, in plain language and without legal conclusions.
- Part 7 shows how to evaluate memory by what the system does across sessions.
The order is deliberate. Each part adds one capability only when the previous design has a specific failure that needs it. That habit, adding memory only when statelessness actually fails, is the most useful thing a team can take into its own design.
Key points
Models are stateless between calls. The application decides what each call can see.
The repeated question is an input problem. A better model given the same input asks the same question.
Working state fixes most continuity failures. A small, typed record, written by the application, deleted when the task ends.
State is not long-term memory. Persisting it beyond the conversation creates a store you then have to govern.
Add memory only when state fails. History, preferences and procedure are different problems with different rules.
Next: Part 2, Conversation history is not a memory strategy. All parts are on the series page.
How CloudNala can help
We help teams that are about to buy or build agent memory work out what they actually need first. Usually that starts with one real conversation flow, the state it depends on and the failures customers see, before any decision about memory products or databases. The result is a design that says what is kept, where, for how long and who may change it.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an Agent Memory Review or write to us at consult@cloudnala.co.za