Most organisations that have been running AI applications for a year or more end up with a gateway. It starts as a convenience: one place to hold the provider keys, one endpoint for developers to call, one dashboard for spend. Then it becomes policy. Routing rules, budgets, redaction, logging and model choice all move into it, because putting them anywhere else would mean repeating them in every application.
That consolidation is sensible. It also turns the gateway into something security teams have dealt with for decades under a different name: a high-trust shared dependency. It holds credentials for paid accounts. It sees every prompt and every response, which means it sees whatever customer and employee data those prompts contain. It has outbound network access by design. And it is usually installed, upgraded and configured by a small platform team moving quickly, often from an open-source project that releases several times a month.
CloudNala's position is straightforward. AI infrastructure is software supply chain. It inherits every conventional risk of a third-party dependency and adds components with unusual access to prompts, credentials, tools and data. Version pinning, provenance, containment and recovery are therefore operating requirements for the gateway and everything like it, not optional hardening for a later phase.
This series works through what that means in practice. This first part sets out the problem and the shape of the answer.
Langeberg Software is fictional, and its systems and figures are illustrative. Langeberg is a Stellenbosch software company of about 140 people. Its main product, Ledgerline, automates accounts payable for mid-size South African firms. Over eighteen months it has added six AI applications, from invoice extraction inside the product to internal coding agents. All six now route through one self-hosted, open-source model gateway running on Kubernetes in a South African cloud region. A platform team of four looks after it.
A real incident made the risk concrete
On 24 March 2026, two releases of LiteLLM, a widely used open-source Python package for routing calls to model providers, were published to the Python Package Index with malicious code in them. According to the project's own security update, versions 1.82.7 and 1.82.8 were live from 10:39 UTC before PyPI quarantined them, and the advisory gives both an "about 40 minutes" figure and a wider window to 16:00 UTC for who may be affected. The advisory describes the payload as a credential stealer: it harvested environment variables, SSH keys, cloud credentials, Kubernetes tokens and database passwords, and sent them to a domain that was not LiteLLM's. Version 1.82.8 also added a .pth file, a Python mechanism that runs code when the interpreter starts.
Three details in that advisory are worth more than the headline.
First, the project believes the compromise came through a dependency in its own CI/CD security scanning workflow. The attacker did not need to find a flaw in the gateway's code. They needed a way to publish.
Second, the advisory states that customers running the official proxy Docker image were not affected, because that path pins its dependencies. The same software, consumed two ways, had two different outcomes. The difference was not the product. It was the install policy.
Third, the recommended remediation was not "upgrade". It was: find every place the bad versions were installed, remove them, rotate every secret those environments could read, and audit pipelines and builds. That is a supply-chain response, and it assumes you can answer "where did this run?" quickly.
We use this incident because it is public and documented by the project itself, not because LiteLLM is unusually risky. Any widely used package in this space is an attractive target for exactly the reason it is useful: it sits in front of everything.
Why the gateway is different from an ordinary library
A compromised logging library in one service is serious. A compromised gateway is a different order of problem, because of where it sits.
Consider what Langeberg's gateway can reach. It holds API keys for two model providers, with spending limits set high enough to avoid throttling the product. It receives the text of supplier invoices, support tickets, draft proposals and fragments of source code. It runs with network access to the internet, because the providers are on the internet. And because six applications call it, a bad release does not affect one team's service. It affects all of them on the next deployment.
This is the core argument of the series. Consolidation concentrates control, and it concentrates exposure in the same place. The architecture decision that makes cost and policy manageable also creates a single component whose compromise would be worth a great deal to an attacker.
The dependencies AI adds
Most engineering organisations have a working answer for conventional supply-chain risk. Packages are scanned. Container images are built in a pipeline and stored in a registry. Some teams produce software bills of materials. That machinery is imperfect, but it exists.
AI infrastructure adds components that mostly sit outside it.
The model gateway is the obvious one, and it often escapes scrutiny because teams think of it as plumbing rather than as an application with secrets.
Models themselves change the behaviour of every answer. A hosted model can change under the same name. A self-hosted model is a large binary file that came from somewhere, and few teams record which file, from where, with which hash.
Prompts and policies decide what the system may do. They are often edited in a configuration screen or a separate repository, outside the release process that governs code.
Agent skills and MCP servers are perhaps the least governed of all. Developers install them from public repositories to make coding agents more capable, usually with whatever permissions they ask for. Some are plain instructions. Others run scripts or hold credentials for real systems. Part 5 of this series treats them properly.
None of these is exotic. Each is a dependency with an origin, a version, an owner and a reach. The problem is that most organisations have not yet given them those properties in any record anyone could consult during an incident.
What "supply chain" means here, in plain terms
Supply-chain security can sound abstract, so it helps to be specific about the questions it asks. For any component that runs in your environment:
- Where did it come from? Which publisher, which repository, which build?
- Is it what it claims to be? Can you check a hash, a digest or a signature against what the publisher released?
- Which version is running, where? Can you answer that for every environment, including CI runners and developer machines, not only production?
- What can it reach? Which secrets, data, networks and tools does it have access to?
- How do you change it? What evidence is required before an upgrade, and how do you go back?
- What happens when it goes wrong? Who decides, how fast can you contain it, and what evidence will still exist?
Public frameworks give these questions structure. SLSA, now at version 1.2, describes levels of assurance for how software is built and, since 1.2, a separate track for source integrity. Bill-of-materials formats such as CycloneDX and SPDX record what a system is made of, and both now have structures for models and datasets, which Part 6 covers. These are useful references. None of them answers the questions for you. They give your answers a consistent shape.
Five commitments for shared AI infrastructure
Langeberg's platform team did not start with a framework. After reading the March advisory, they asked whether they could prove their own exposure, and found they could prove it for production but not for everything else. What they adopted afterwards fits in five commitments, and each one gets its own part in this series.
- Pin every version, and upgrade deliberately. Production, CI and developer tooling install exact versions with hashes. Upgrades walk a defined path with tests and a tested rollback. (Part 2)
- Know the containment sequence before you need it. Identify, isolate, preserve, rotate, verify, rebuild, monitor and record, in that order. (Part 3)
- Keep the evidence that answers "were we exposed?" Registry history, digests, CI logs, egress logs and runtime logs, with retention chosen on purpose. (Part 4)
- Treat skills, MCP servers, prompts and models as inventory. Give each a version, a source, an owner and a record of what it can touch. (Parts 5 and 6)
- Design containment in. The gateway runs with scoped identities, short-lived secrets, an egress allow-list and a kill switch, so a compromise is bounded even when it is not prevented. (Part 7)
The last two parts deal with the record that ties this together and with how to judge open-source risk on evidence rather than stereotype.
What this is not
It is not an argument against gateways. A well-run gateway is one of the better tools an organisation has for controlling AI cost, applying policy and seeing what its applications do. The Agent Evidence Layer article The gateway is a control point, not a trust boundary makes the case that a gateway is a control point rather than a trust boundary, and The Model Gateway Is Not the AI Platform series covers what a gateway should and should not be asked to do. This series is about the gateway, and its neighbours, as dependencies.
It is also not an argument against open source. The March incident was resolved by an open-source project publishing a detailed, dated advisory with indicators of compromise and remediation steps. Part 9 returns to this: the useful question is not "open source or vendor?" but "what evidence do we have about this component, and how is it contained?"
And it is not a reason to stop. The organisations most exposed to this class of risk are not the ones moving fastest with AI. They are the ones that consolidated onto shared AI infrastructure without moving their supply-chain discipline across with it.
A first check you can run this week
Pick your most shared AI component, whether that is a gateway, an orchestration framework or an agent runtime, and answer four questions from evidence rather than memory:
- Which exact version runs in production, in CI and on developer machines?
- Is that version pinned by hash or digest, or resolved at install time?
- Which secrets could its process read if it were compromised?
- If an advisory named a bad version tomorrow, how long would it take to prove whether you ever installed it?
If the answer to the fourth question is "we would need to ask around", the rest of this series is for you.
Next: Part 2, Pin the version before you pin your hopes on the platform. All parts are on the series page.
How CloudNala can help
We review shared AI infrastructure as a supply-chain question: what each component is, where it came from, how it is pinned and upgraded, what it can reach and how a compromise would be contained. The output is a short inventory of AI dependencies, a ranked list of exposure gaps and a containment runbook the platform team can rehearse, rather than a generic policy document.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Supply Chain Review or write to us at consult@cloudnala.co.za