Every part of this series so far has been about keeping bad components out, or finding them quickly when they get in. Pinning, review gates and inventories reduce the chance of compromise. None of them reduces it to zero. A maintainer account can be taken over, a signed release can carry a backdoor, a reviewed skill can be updated badly.
So the remaining question is architectural: when something malicious does run inside your AI infrastructure, how much can it do? That answer is decided long before the incident, by how the gateway and its neighbours were deployed.
Langeberg Software is fictional, and its systems and figures are illustrative. Langeberg's original gateway deployment was typical of a platform that grew quickly: one container with long-lived provider keys in environment variables, a single key per provider shared by all six applications, unrestricted outbound internet access and a writable file system. It worked well. It would also have handed an attacker everything at once.
CloudNala's position is that containment is a design property, like availability. It should be specified, built and tested, not added after the first incident.
The controls that bound a compromise
Langeberg rebuilt the gateway deployment around six controls. Each one limits what a compromised gateway could do, independently of how it was compromised.
Its own identity, with least privilege
The gateway runs under a dedicated service identity that can read its own secrets and write its own logs, and nothing else. It cannot list other workloads' secrets, read storage buckets or call the cloud control plane. Kubernetes service account tokens are not mounted unless needed. This is the same least-privilege principle discussed in Identity before intelligence, applied to the gateway's own process rather than its callers.
Short-lived secrets from a secrets manager
Provider keys come from a secrets manager at start-up and are rotated on a schedule, rather than living in environment variables baked into a deployment manifest. A stolen key therefore has a shelf life. Where a provider supports scoped or workload-identity credentials, those are preferred over long-lived API keys.
An egress allow-list
The gateway may open connections to exactly two destinations: the two providers' API endpoints. Everything else fails closed and raises an alert. Applied to the CI runner where it ran, the same kind of control would have neutralised the payload in the Part 3 scenario, which needed to reach a host that was neither a provider nor the registry. It is the cheapest high-value control in this list, and the one most often missing, because "the gateway needs the internet" is true only for a very short list of hosts.
The same principle applies to build infrastructure. CI runners that install packages need the registry or, better, an internal mirror. They rarely need the rest of the internet.
A hardened runtime
The container runs with a read-only root file system, as a non-root user, without a shell, and with only the capabilities it needs. A payload that wants to persist, install tools or modify start-up files has much less to work with.
Per-application keys and budgets
Each consuming application authenticates to the gateway with its own key and has its own budget and rate limit. Upstream, where providers allow it, separate projects or keys per application keep usage attributable.
A kill switch
One configuration flag stops all routing through the gateway, returning a clear error to applications. It is tested quarterly. When the question is "is the gateway itself compromised?", the ability to stop it in one change, without a deployment, is what lets the team isolate first and investigate second.
The difference scoping makes
Per-application scoping deserves its own picture, because it changes two things at once.
It caps the damage. With one shared provider key, a stolen credential can spend against every provider account the organisation holds, up to whatever limit was set to avoid throttling production. With per-application keys and budgets, theft is bounded to one application's quota.
It makes abuse visible. Spend on a shared key is a single line that grows for many reasons. Spend on a per-application key has its own baseline, so an anomaly stands out. The Model Gateway Is Not the AI Platform series covers keys as an accountability mechanism in more depth; here the point is containment.
Test the containment, not just the configuration
A control that has never been exercised is a hope. Langeberg tests three things on a schedule:
| Test | How | Pass condition |
|---|---|---|
| Egress allow-list | From inside the gateway pod, attempt a connection to an unlisted host | Connection refused, alert raised within minutes |
| Kill switch | Flip the flag in staging, then production during a quiet window | All AI calls fail cleanly, applications degrade as designed, restore in one change |
| Secret rotation | Rotate a provider key without warning the team | No outage; old key confirmed revoked |
The kill-switch test also tests the applications. Each one should degrade in a designed way when the gateway is off: the invoice extraction falls back to a manual review queue, the support assistant shows a "temporarily unavailable" message and routes to a person. An application that crashes when the gateway is unavailable has turned a containment action into an outage.
Trade-offs worth naming
Containment has costs, and they are worth stating plainly.
Operational friction. An egress allow-list must be updated when you add a provider or a provider changes its endpoints. Short-lived secrets need a working secrets manager and rotation that is genuinely automated.
Some features need more reach. A gateway that calls web search or fetches URLs on behalf of agents cannot have a two-host allow-list. That function belongs in a separate, more tightly sandboxed component, not in the gateway that holds every provider key.
Debugging is harder in a locked-down container. No shell means investigating through logs and ephemeral debug containers. That is a habit worth building anyway.
None of these is a reason to skip containment. They are reasons to design it deliberately, so the friction falls where it buys the most.
The test
Imagine a malicious version of your gateway running in production for an hour. Write down what it could read, where it could send it, how much it could spend and how you would stop it. If the answers are "everything", "anywhere", "a lot" and "a redeployment", containment is the next piece of work.
Next: Part 8, Upgrade evidence belongs in the architecture record. All parts are on the series page.
How CloudNala can help
We review how shared AI infrastructure is deployed and design the containment around it: identity, secrets, egress, runtime hardening, per-application scoping and a tested kill switch. The outcome is a deployment where the answer to "what could a compromised gateway do?" is short and written down.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Supply Chain Review or write to us at consult@cloudnala.co.za