Most model discussions start with capability: which model reasons best, writes best or follows instructions best. For a South African bank, that question comes second. The first question is where each kind of work is allowed to be processed, and what the platforms actually offer within those limits. The answer shapes the architecture more than any benchmark does.
Chapter 8 is also where a design that has been neutral about models so far has to commit. Impala Bank's assistant has used a hosted model since Chapter 2, through the bank's Azure landing zone in South Africa North. As the assistant started handling signed-in conversations, the bank's privacy and risk teams asked the question that should be asked early in every such programme: where, exactly, does the inference run?
Where we are in the case study
Impala Bank is fictional, and its systems and figures are illustrative. After Chapter 7, Ask Impala answers public and account questions, uses six read-only tools, takes three bounded actions and opens cases in the bank's existing workflows for disputes and complaints. All of that sits behind one model box in the architecture. This chapter turns that box into a gateway.
What "available in South Africa" means today
We checked the published documentation on 14 October 2026. These details change often, so treat them as a dated snapshot and recheck before you decide anything.
Microsoft Foundry in South Africa North. Microsoft's deployment-type guidance distinguishes Global types, which "may be processed in any Azure region", Data Zone types, which process data only within a Microsoft-specified zone (US, EU or Asia Pacific), and Standard and Regional Provisioned types, which process prompts and responses within the customer-specified Azure geography. Microsoft's region availability tables for South Africa North list current GPT models under Global Standard. Under regional Standard, they list embedding models only. Under Regional Provisioned, they list a single 2024 version of gpt-4o. Data Zone deployment is shown as not available for the Middle East and Africa tab, and there is no African data zone.
Amazon Bedrock in Cape Town. Bedrock's model-by-region tables show the current Claude models in af-south-1 under global cross-Region inference only, with in-Region support marked as not supported for those models.
Put plainly: if Impala wants a current frontier chat model, through either major hyperscaler, its requests will be processed outside South Africa. If it insists on in-country inference from those platforms today, its choice of chat model is narrow and dated. Our earlier reference article, AI models available in South Africa, explains the difference between "available in" and "processed in" in more depth.
This is a design question, not a prohibition
It is important not to overcorrect. POPIA does not forbid sending personal information outside South Africa. Section 72 sets conditions for cross-border transfers, the most commonly relied on being that the recipient is bound by law, binding corporate rules or a binding agreement that provides an adequate level of protection. Many South African organisations use global cloud and AI services lawfully on that basis. Our explainer POPIA does not require your data to stay here sets out the five conditions in plain language.
Banks, however, carry obligations beyond POPIA: their own outsourcing and data-governance policies, the expectations of their regulators, contractual commitments to clients and their own risk appetite. For Impala, the question was not whether global processing is ever lawful. It was which of its conversations it was comfortable sending offshore, under what conditions, and what it would do with the rest. That is a decision for the bank's privacy, legal and risk functions, informed by the architecture. This chapter is general information, not legal advice.
Three paths, honestly compared
Impala's architects set out three paths.
Hosted, global. Current models, lowest effort, priced per token, and processing in whichever region the provider chooses. The bank depends on contractual and technical assurances about how prompts are handled and retained.
Hosted, in South Africa. Processing stays in the country, but on the platforms we checked the choice of chat models is very limited today. It suits embeddings for search well, and little else for a conversational assistant.
Self-hosted in South Africa. An open-weight model running on GPU capacity Impala controls, in an in-country cloud region or its own data centre. The bank chooses the model and keeps processing local. It also takes on everything the providers normally carry: capacity planning, patching, security, scaling for month-end peaks, model upgrades and a 24-hour operations rota. GPUs cost money whether they are busy or idle, and capacity in-country should be confirmed rather than assumed.
None of the three is right for everything. That observation led to the decision that defines this chapter.
Route by the data the work carries
Instead of choosing one model for the whole assistant, Impala put a small model gateway between the orchestrator and the models, and routed each kind of work by the data it carries.
Every incoming message first goes to a small self-hosted model in South Africa. Its jobs are narrow: classify the intent, detect personal details such as ID numbers, account numbers and addresses, and mask them. Smaller open-weight models handle classification and detection well, and these tasks need no frontier reasoning. Because this step runs before anything leaves the country, the raw message never does.
Public questions then use a hosted global model freely. By definition they carry no client data beyond the question itself, which has already been masked.
Signed-in conversations use the hosted model only with minimised, masked context, and only after Impala's privacy and risk functions approved that use, the provider's terms on data use and retention were reviewed, and the cross-border basis was documented. The account slices from Chapter 3 were already designed to be small; here that design pays off a second time.
Out-of-scope or sensitive conversations use no model at all. Hardship, bereavement and fraud are recognised by the in-country classifier and routed straight to people, as in Chapter 7.
Embeddings for the public knowledge index run on an in-country embedding deployment, since those are offered locally and the content is public anyway.
What the gateway gives the bank
The gateway is more than a router. It became the single place where Impala could enforce model policy: which deployments each kind of work may use, token budgets per conversation, timeouts and fallbacks when a provider is slow or unavailable, and a record of every model call for Chapter 10's cost and evaluation work.
It also gave the bank exit options. If an in-country deployment of a suitable current model becomes available, signed-in conversations can move to it by changing a routing rule, not by rebuilding the assistant. If a provider changes its terms, the gateway can shift traffic. Model choice became a configuration decision with a review process, rather than an architectural commitment.
What could go wrong
The design has costs worth naming. The self-hosted classifier is now a critical dependency: if it fails, nothing reaches the hosted model, so it needs the same availability engineering as any core service, and a fail-safe that routes to people rather than skipping masking. Masking is imperfect: an unusual way of writing an account number may slip through, which is why the hosted path also receives only pre-shaped account slices rather than raw conversation history. And two model paths mean two sets of evaluations, since a change on either side can alter behaviour.
Impala judged those costs worth paying for a design that keeps raw client messages in the country, uses capable models where the data allows, and can change as the market changes.
Key points
Check what is offered, and date it. On 14 October 2026, current chat models from the major hyperscalers were processed globally when called from South Africa.
Global processing is a decision, not a prohibition. POPIA section 72 sets conditions; the bank's own obligations and risk appetite decide the rest.
Each path has a real cost. Global hosting cedes location, in-country hosting limits choice, self-hosting adds operations.
Route by the data the work carries. Classify and mask in-country first, then send only what each path is approved to receive.
A gateway turns model choice into configuration. Policy, budgets, fallbacks, records and exit options live in one place.
Next: Chapter 9, Guardrails are a stack, not a switch. All chapters are on the series page.
How CloudNala can help
We help South African organisations choose model paths with evidence: what each platform actually offers in-country today, how different kinds of work should be routed, what a gateway needs to enforce, and what self-hosting will really cost to run. We work alongside your privacy, legal and risk teams rather than around them.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Production Readiness Review or write to us at consult@cloudnala.co.za