The Production AI Case Study
A bank's contact centre is flooded with repetitive questions whose answers already exist in its app and website. The goal is better access and lower service cost without giving an AI system unsafe authority over accounts or payments. This ten-chapter case study grows one architecture step by step: define the outcome and baseline, start with public information, add authenticated context, separate public knowledge from private data, introduce read-only tools, add bounded actions, decide where one agent stops and workflows begin, choose hosted or self-hosted model paths, layer guardrails and escalation, and put evaluation, observability, cost and ownership in place. The bank is fictional and every figure is illustrative.
10 articles
- Part 1Define the outcome before you build the assistantMost bank assistants are judged by how many conversations they deflect, which rewards the wrong thing. Chapter 1 of the Production AI Case Study starts where production starts: a business outcome, a measured baseline and a short list of things a cheaper channel is not allowed to make worse.8 min read
- Part 2Start with an assistant that knows nothing about anyoneThe safest first assistant is one that cannot see any client's data at all. Chapter 2 of the Production AI Case Study builds that version: answers only from approved public content, a route to a person, and no guessing, and shows why such a modest start teaches a bank most of what it needs to know.8 min read
- Part 3Adding account context without handing the model the keysThe most common question in Impala Bank's contact centre is about the client's own account, and the first assistant could not answer it. Chapter 3 of the Production AI Case Study adds identity and a narrow slice of account context, and shows why the session, not the model, decides whose data can be seen.8 min read
- Part 4Separate public knowledge from private account dataOnce an assistant can see account data, the quickest design puts help pages and client records in one searchable store and relies on filters to keep them apart. Chapter 4 of the Production AI Case Study draws a hard boundary instead, and shows what it changes for storage, logging, retention and testing.8 min read
- Part 5Read-only tools: letting the assistant look, not touchTools turn an assistant that answers into one that can look things up. Chapter 5 of the Production AI Case Study gives Impala Bank's assistant its first read-only tools, and shows why a tool's contract, scope and limits matter more than how clever the model is at choosing it.8 min read
- Part 6Bounded actions: what the assistant may do, and what it never willThe first time an assistant changes something, the questions change from accuracy to authority. Chapter 6 of the Production AI Case Study lets Impala Bank's assistant request a confirmation letter, freeze a card and order a replacement, and draws a permanent line around payments, payees, limits and contact details.8 min read
- Part 7Where one agent stops and workflows beginAs an assistant gets more capable, the pressure grows to let it handle disputes, fraud reports and complaints too, or to split it into many cooperating agents. Chapter 7 of the Production AI Case Study argues for a different boundary: the assistant starts the case, and the bank's existing workflows and people own it.8 min read
- Part 8Hosted or self-hosted: choosing a model path from South AfricaFor a South African bank, the model decision is less about which model is best and more about where each kind of work may be processed. Chapter 8 of the Production AI Case Study checks what the major platforms actually offer in-country today, compares three model paths, and routes Impala Bank's work by the data it carries.9 min read
- Part 9Guardrails are a stack, not a switchGuardrails are often sold as one product that makes an assistant safe. Chapter 9 of the Production AI Case Study pulls Impala Bank's controls into a deliberate stack of input, retrieval, tool, policy, output and human layers, and shows what each catches, what each misses and why escalation to a person is a control rather than a failure.9 min read
- Part 10Evaluation, observability, cost and ownership: running it for realAn assistant is not finished when it launches; it is finished when the organisation can prove, every month, that it still works, what it costs and who answers for it. Chapter 10, the last in the Production AI Case Study, closes the loop back to the Chapter 1 scorecard with evaluations, traces, cost per resolved question and a named operating rhythm.9 min read
Working through any of this in your own organisation?
These articles come out of delivery work rather than theory. If the problems in this series look familiar, a short scoping conversation is usually the quickest way to work out what is worth doing first.
Talk to CloudNala