Ask three people what an API gateway is and you will usually hear three different answers. The network engineer says it is a reverse proxy. The security lead says it is where authentication happens. The vendor says it is a platform that will give you visibility, monetisation and governance across your entire digital estate.
All three are partly right, which is exactly why gateway decisions go wrong. Teams buy one because a reference architecture had a box with that label, or they avoid one because "it is just another hop", and in both cases nobody has said which problem it is meant to solve.
This series, Architecture in Plain English, takes architecture decisions that are usually explained in jargon and explains them through the trade-off underneath. It starts with the gateway because it is one of the most common boxes on a diagram and one of the least examined.
Marula Commerce is fictional, and its systems and figures are illustrative. Marula is a Pretoria online homeware retailer. Its shop platform, a single application built in 2017, now sits behind a website, a mobile app, two marketplace partners that list Marula's products, and a courier that sends delivery updates back by webhook. Each of those consumers talks to Marula's APIs, and each one was connected by a different developer at a different time.
The plain-English definition
An API gateway is a single, managed entry point that every external request passes through before it reaches the service that does the work. Its value comes from doing the same handful of checks consistently, in one place, for every API behind it.
The CloudNala view is simple: a gateway is a control point, not a network hop. If you cannot name the controls you want applied consistently across several consumers, you do not yet need one. If you can, a gateway is usually cheaper than rebuilding those controls in every service.
The five jobs a gateway does
Authenticate. The gateway checks who is calling, usually by validating a token issued by an identity provider or an API key tied to a known partner. A backend that sits behind a gateway can trust that an unauthenticated request never reached it. It still has to decide what the authenticated caller may do with a specific record, because that rule depends on the data, and the data lives in the backend.
Apply policy. This is the job that carries the most value and gets the least attention. Policies are rules such as "marketplace partners may read catalogue data but never customer data", "requests larger than 1 MB are rejected", or "this API version is deprecated and returns a warning header". Written once at the gateway, they apply everywhere. Written in each service, they drift.
Throttle. Rate limits protect backends from a misbehaving client and protect paying consumers from each other. When Marula's second marketplace partner ran a full catalogue sync every five minutes, it slowed the shop for everyone. A per-consumer limit at the gateway would have returned 429 Too Many Requests to that partner and left shoppers alone.
Route. The gateway maps a public path to an internal service and version. This is what lets Marula move the catalogue out of the shop platform later (the subject of Part 2) without asking partners to change their integration.
Observe. Because every request passes through it, the gateway is a natural place to record who called what, how long it took and what came back. That gives one consistent view of traffic across consumers, which is otherwise surprisingly hard to assemble.
Most products add more: request and response transformation, caching, a developer portal, API key self-service, analytics and billing. These are useful, but they are not why you adopt a gateway. If the five core jobs do not justify it, the extras will not.
What a gateway does not do
The most expensive gateway mistakes come from assuming it does more than it does.
It does not make an API secure on its own. It authenticates callers and enforces coarse rules. It cannot know whether customer 4471 may see order 9182. That check belongs in the orders service. A gateway in front of a backend with no authorisation logic is a locked front door on a house with open windows.
It does not govern traffic that bypasses it. If an internal job calls the orders database directly, or a developer gives a partner a direct service URL "just for testing", the gateway sees none of it. Its controls cover only the paths that go through it, so the architecture has to make it the only path, usually with network rules that block direct access to backends.
It is not a load balancer, though it often sits next to one. A load balancer spreads traffic across healthy instances of the same service. A gateway decides whether a request should reach a service at all, and which one. Cloud platforms often bundle both, which blurs the line. Our load balancing and TLS article covers the load balancer's side.
It is not a service mesh. A mesh manages traffic between internal services: mutual TLS, retries, traffic shifting. A gateway manages traffic coming in from outside. Some organisations need both, most start with neither, and very few need a mesh before they need a gateway.
It is not an AI gateway or an MCP gateway, though the ideas rhyme. An AI gateway sits between applications and model providers and is concerned with model routing, token budgets and prompt logging. An MCP gateway mediates agent access to tools. Both borrow the control-point idea and add problems of their own. We cover them in The Model Gateway Is Not the AI Platform and The enterprise MCP gateway.
The trade-off: what you pay for consistency
A gateway is not free, and the costs are mostly operational rather than financial.
| Cost | What it means in practice |
|---|---|
| Latency | A few milliseconds per request in a well-run gateway; more if policies call out to other services |
| A new single point of failure | If the gateway is down, every API is down. It needs redundancy and its own on-call |
| An owner | Someone must review policy changes, manage certificates, onboard consumers and retire versions |
| Configuration as code | Gateway rules changed by hand in a console drift just like servers do |
| Skills | Policy languages differ across products and need people who understand them |
The most common failure is the second-to-last row turning into the third. A gateway is installed by a project, the project ends, and two years later nobody can say why a particular rule exists or whether it can be removed. Treat the gateway as a product with a named owner from the first day.
When you need one, and when you do not yet
The decision rests on two questions.
Is there more than one consumer outside the team that builds the API? If Marula's only consumer were its own website, built by the same team that builds the shop platform, the controls could live in the application and a load balancer would do. The moment partners, a mobile app and a courier all connect, consistency starts to matter.
Do the same controls need to apply across several APIs? If every API needs the same authentication, the same rate-limit model and the same logging, a gateway removes a large amount of duplicated work. If each API is genuinely different, a gateway mostly adds a hop.
Marula answered yes to both. Its first gateway release had a deliberately short scope: token validation for the app, API keys and per-partner rate limits for the marketplaces, a signed-webhook check for the courier, and one dashboard for all traffic. Caching, a developer portal and request transformation were left for later, because nobody could yet name the problem they solved.
South African considerations
Two local conditions shift the decision slightly. First, many organisations here integrate with partners that have limited engineering capacity, such as smaller couriers, municipal systems and regional retailers. A gateway that can absorb some of their awkwardness (a legacy authentication scheme, an odd payload format) at the edge can save months of negotiation. Second, where personal information flows to partners, POPIA keeps the responsible party accountable for that information, including where an operator processes it on its behalf, and that is hard to demonstrate without knowing which partner receives what. A gateway that logs every partner call against a named consumer makes that question answerable. It does not, on its own, make the flow lawful.
Managed service, open source or both?
Every major cloud offers a managed gateway, and several mature open-source gateways exist that can run anywhere. The choice matters less than teams expect and is usually settled by three practical questions. Where do most of your backends run? A gateway close to them avoids routing traffic out of one environment and back into another. Who will operate it? A managed service removes patching and scaling work but ties policy definitions to that provider's format. How portable do your policies need to be? If you expect to run across several clouds or on-premises, an open-source gateway with policies kept in version control is easier to move.
For most South African organisations starting out, a managed gateway in the environment where their main backends already run is the pragmatic default. The policies, consumer list and ownership model are what carry over if that ever changes, so invest in those rather than in the product choice.
A short checklist before you buy or build
- List every consumer of your APIs today and in the next twelve months.
- Write down the controls you want applied consistently. If the list is shorter than three items, wait.
- Name the owner of the gateway as a product, not as a project deliverable.
- Decide how backends will be protected from direct access.
- Keep authorisation of specific records in the backend services.
- Put gateway configuration in version control from day one.
- Start with the five core jobs; add features only against a named problem.
A gateway is one of the few architecture components whose value grows with the number of things connected to it. That is also why its weaknesses spread so far. Choose it for the controls it gives you, and design for what it cannot see.
Next: Part 2, When a module should become a microservice. All parts are on the series page.
How CloudNala can help
We help teams decide whether a gateway is warranted, which controls belong at the edge and which belong in services, and how to run the gateway as an owned product. The output is a short API control map, a consumer inventory and a set of policies ready to be written as code.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Request an Architecture Review or write to us at consult@cloudnala.co.za