Ask a vendor how to make an assistant safe and the answer is often a single product: a guardrail service that sits in front of the model and blocks bad things. Those services can be useful. The idea that one layer can make an assistant safe is the problem. Each control catches some failures and misses others, and attackers, confused clients and unusual situations will find whatever a single layer misses.
By Chapter 9, Impala Bank already had many controls, added one chapter at a time: identity from the session, masked account slices, a public-private boundary, read-only tools with limits, native confirmations for actions, workflows that own cases, and an in-country classifier. This chapter does two things. It names the stack those controls form, so the bank can reason about gaps, and it adds the layers that were still missing: a deterministic policy engine and a deliberate escalation design.
Where we are in the case study
Impala Bank is fictional, and its systems and figures are illustrative. After Chapter 8, every message to Ask Impala is first classified and masked by a small self-hosted model in South Africa, then routed through a gateway: public questions to a hosted global model, signed-in conversations to it only with minimised context, and sensitive conversations straight to people.
Six layers, each with a different job
Impala's architects described the stack in six layers. The point of naming them was not tidiness. It was to make each layer's blind spots explicit, so no one could believe that a strong layer excused a weak one.
Input. Checks what arrives: masking personal details, recognising fraud and distress language, and stripping or neutralising instructions hidden in pasted text, such as "ignore your previous instructions and…". It cannot know whether a perfectly polite request is being made by a fraudster who has taken over a client's device.
Retrieval. Controls what knowledge the assistant can draw on: approved public content only, and account data only through session-checked services. It cannot stop the assistant from misreading a correct page.
Tool. Controls what the assistant can reach: an allowlist of tools, the client taken from the session, validated inputs and call limits. It cannot judge whether a permitted action makes sense in context.
Policy. Decides, deterministically, whether a proposed action may proceed. More on this layer below, because it was the one Impala added in this chapter.
Output. Checks what leaves: no unmasked numbers, no statements that read as financial advice, no promises the bank has not approved ("your money will be refunded"), and links to the source for public answers. It cannot detect an answer that is fluent, plausible and wrong because the underlying page was out of date.
Human. Brings a trained person in when the situation needs judgement, empathy or authority the assistant does not have. It cannot scale to every conversation, which is why it must be targeted well.
Policy belongs in code, not in a prompt
Before Chapter 9, the rules about actions were spread across tool code, confirmation screens and the model's instructions. The instructions said things like "only order a replacement card if the client's current card is lost, stolen, damaged or expired". That is a business rule expressed in a prompt, and a model follows prompts most of the time, not all of the time.
Impala moved every such rule into a small policy service that the orchestrator must consult before any action. The service receives the proposed action, the client's relevant state from the bank's systems and the context of the request, and returns allow, deny or escalate, with a reason. For example, illustratively:
- A replacement card is allowed if the current card is reported lost, stolen, damaged or expired, and no other replacement was ordered in the last 14 days. Otherwise escalate.
- A card freeze is always allowed, because it reduces risk.
- Any action is denied if the device was registered in the last 24 hours, a common sign of account takeover. The client is routed to a person.
- Any action is escalated if the in-country classifier has flagged fraud or coercion language anywhere in the conversation.
These are deterministic rules, written and owned by the card and fraud teams, versioned and tested like any other code. The model proposes; the policy decides. This is the same separation our Jev: The Decision Layer series describes between a signal and the rules that act on it, and the same idea as the tool gateway as a control point in the Agent Evidence Layer series.
Escalation is a control, not a failure
Many assistant programmes measure escalation to a person as a cost to minimise. Impala treated it as a control to design, with the same care as any other.
Escalation triggers fell into four groups. The client asks: any request for a person is honoured, immediately, without the assistant trying to persuade them to stay. The situation is sensitive: signs of vulnerability, bereavement, hardship, fraud or coercion, recognised by the classifier or by the conversation's content. The assistant is failing: the same question asked three times, two consecutive tool failures, or an answer the client says is wrong. Policy says so: an escalate decision from the policy service.
The quality of escalation mattered as much as its timing. The contact centre received the conversation, the reason for escalation and any policy decision, so the person picking it up knew immediately why they were involved. Fraud and vulnerability escalations went to the front of the queue. And the assistant told the client honestly what was happening: "I'm connecting you to a colleague who can help with this. They'll see our conversation, so you won't need to repeat yourself."
Our AI Before the Rulebook series argues that human oversight only counts if the human can actually see and question what the system did. Escalation designed this way gives them that.
Testing the stack, not just the model
Testing guardrails by asking the model to misbehave and checking that it refuses tests only one layer. Impala's red-team exercises targeted each layer and, more importantly, the gaps between them.
A few of the scenarios, illustratively: a client pastes a message claiming to be from Impala's fraud team instructing the assistant to "verify" the account by reading out recent transactions. A device registered two hours earlier asks politely for a replacement card to be delivered. A client in obvious distress asks only about fees. A help page is deliberately made stale in a test environment to see whether the output layer catches the contradiction (it did not, which is why content ownership from Chapter 2 remains the real control for that failure).
Each scenario had an expected outcome at the layer that should catch it, and a check that the next layer would still have held if that one failed. Where no layer caught a scenario, a control was added or the assistant's scope was reduced.
What guardrails cannot do
Guardrails do not make an assistant safe. Scope does most of that work, and Impala's scope decisions, no payments, no payees, no limit changes, no contact-detail changes, no case decisions, removed the highest-harm failures before any guardrail was needed. Guardrails reduce the probability and impact of failure inside a scope already chosen to be survivable.
Saying that plainly to the board mattered. It stopped the guardrail stack from being used as an argument for widening scope, the pattern where each new control is cited as the reason the next risky capability is now acceptable.
Key points
No single layer makes an assistant safe. Input, retrieval, tool, policy, output and human layers each catch different failures.
Name each layer's blind spots. A strong layer does not excuse a weak one.
Put business rules in a deterministic policy service. The model proposes; versioned rules owned by the business decide.
Design escalation as a control. Clear triggers, priority routing, full context and an honest message to the client.
Scope does most of the safety work. Guardrails reduce risk inside a scope already chosen to be survivable.
Next: Chapter 10, Evaluation, observability, cost and ownership. All chapters are on the series page.
How CloudNala can help
We help organisations design guardrails as a stack: what each layer must catch, which rules belong in a deterministic policy service, how escalation should work, and how to red-team the gaps between layers rather than only the model. We are honest about what guardrails cannot do, so they are not used to justify scope your risk appetite does not support.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Production Readiness Review or write to us at consult@cloudnala.co.za