Identity before intelligence: who may call the model, and on whose authority

An API key proves that the caller has a secret. It does not prove who the caller is, which employee started the request, which application is responsible, or which business action is allowed. Yet many first AI deployments run on exactly that: one key, pasted into an application's configuration, shared by every user and every feature.

Enterprise AI involves several identities at once: the human user, the client application, the orchestration workload, the Foundry project, the agent, the tool or downstream API it calls, and the operator who changes any of them. Collapse those into one shared key and two things become very hard: least privilege, because everything has the key's full power, and investigation, because every request looks the same.

This part is about keeping those identities distinct, deciding what a shared AI gateway should enforce, and settling a question that matters more in South Africa than almost anywhere else: where does the model actually process the request?

Fynbos Health Administrators is fictional, and its systems and figures are illustrative. Fynbos, the medical scheme administrator from Part 1, runs three AI products: the internal Policy Assistant, the Member Desk copilot for contact-centre agents, and the Claims Query agent. The Policy Assistant started life with an API key in its configuration. This part removes it.

The identity chain

The goal is an accountability chain that survives every hop, from the person who asked to the system that acted.

At Fynbos, a contact-centre agent signs in to the Member Desk app with Entra ID. The app calls the AI gateway with its own managed identity plus the signed-in user's context. The gateway checks that the app may use the requested model and forwards the call. The model returns a proposed answer or action. If an action is needed, the app, not the model, calls the claims API, scoped to the one member verified on the call, and records the result with an audit reference. The exact implementation will vary. What should not vary is that, at every step, someone can say who asked and under whose authority the system acted.

Control plane and data plane

Foundry separates management operations from development and runtime operations. Creating projects or model deployments is a control-plane action. Building agents, uploading files, running evaluations and invoking models are data-plane actions. Entra ID and Azure role-based access control apply at both the Foundry resource and project scopes, as Microsoft's Foundry RBAC documentation explains.

At the time of writing (October 2026), Microsoft has renamed the built-in roles: Foundry User, Foundry Owner, Foundry Account Owner and Foundry Project Manager replace their "Azure AI" predecessors, and there is a new Foundry Agent Consumer role. Microsoft also advises against using the older Azure AI Developer role for Foundry, because it was designed for hub-based and Azure Machine Learning workspaces. Check the current names before writing them into a design; they have changed once and may change again.

Use that structure to define personas rather than individuals:

Assign roles to groups and managed identities at the narrowest practical scope, and avoid permanent individual assignments for privileged roles. Fynbos's Member Desk developers hold their roles on their own project only; nobody in a product team holds Foundry Account Owner on the member-data resource.

Managed identity first

Where an Azure service supports Entra authentication, use a managed identity. It removes long-lived secrets from code and pipelines. It does not remove the need to design permissions. For every managed identity, record:

• the resource or workload it belongs to;
• the roles assigned, and at exactly which scopes;
• the data-plane permissions it needs;
• whether it is system-assigned or user-assigned;
• who approves changes to it;
• how its access is reviewed and removed.

A managed identity with Owner rights or broad data access is still over-privileged. It is just over-privileged without a password.

Fynbos removed the Policy Assistant's API key in the reorganisation. The app now authenticates with a managed identity, local key authentication is disabled on the Foundry resource, and the identity's role assignment is recorded alongside the product's other decisions.

What the AI gateway should do

Azure API Management can provide one governed endpoint in front of several models and tools. Its AI gateway capabilities are delivered as policies, including llm-token-limit for token quotas, llm-emit-token-metric for usage metrics, llm-content-safety for content checks, and semantic cache lookup and store policies. Backend pools (round-robin, weighted, priority or session-aware) and a circuit breaker that honours a backend's retry instructions support routing and failover. Microsoft documents these under AI gateway capabilities in API Management.

One distinction matters for a production design. At the time of writing, the dedicated AI Gateway tier of API Management is in public preview, in a small number of regions and without an SLA, and the AI gateway integration inside Foundry is also in preview. The policies above, running on the standard API Management tiers, are the production path today. Check the status again before committing.

Used well, the gateway can enforce:

• consumer authentication and authorisation;
• model allow-lists per product;
• per-product or per-tenant token limits, request quotas and rate limits;
• model routing and backend failover;
• standard request metadata and correlation identifiers;
• content-safety checks;
• usage metrics for allocation and FinOps;
• protection of provider credentials from applications.

Fynbos's AI platform team runs the gateway for all three products, each with its own subscription, its own model allow-list and its own token limit, so one product's runaway loop cannot exhaust another's capacity, and the monthly usage report splits cleanly by product for chargeback.

What the gateway cannot do is just as important. It cannot decide whether an answer about a hospital co-payment is correct, whether a retrieved document should have been visible to this user, or whether a goodwill credit needs a team leader's approval. Do not present it as the whole AI governance layer. It applies shared runtime policy; the product decides what a request means.

Model routing needs policy

Routing between models can improve availability, cost or speed. But a fallback model is not automatically equivalent to the primary one. Before routing between them, check each of these against the product's requirements:

• data-processing location;
• contractual and licensing terms;
• input and output limits;
• tool-calling behaviour and safety-filter behaviour;
• supported languages;
• evaluation results on the actual use case;
• latency and throughput;
• retirement dates. At the time of writing, Microsoft gives at least 60 days' notice before retiring a generally available model, and retirement dates cannot be extended.

A cheaper fallback that fails the product's quality threshold is not resilience. It is a different failure mode, arriving at the worst possible moment. Fynbos allows failover only between deployments of the same model version, and never to a deployment with a different processing scope.

Semantic caching is a data decision

Semantic caching stores previous prompts and responses and returns a cached answer when a new question means roughly the same thing. It can cut repeated model calls and latency. In API Management it needs Azure Managed Redis or another RediSearch-compatible cache plus an embeddings deployment, so it is also a new store of inputs, outputs or their representations, with its own privacy and correctness questions:

• Can a cached response cross user, tenant or permission boundaries?
• What is in the cache key?
• How are policy or knowledge changes invalidated?
• What is the retention period?
• Does the response include time-sensitive information?
• Could an unsafe or wrong answer be amplified by reuse?

Fynbos's answer was different for each product. For the member products, caching is off: a cached answer about one member's benefits could too easily be served to another. For the Policy Assistant, the team designed a narrow version, limited to general policy questions, partitioned by audience group and invalidated whenever the index version changes, and then deferred it, because the assistant's volume did not justify a new data store. Do not enable semantic caching only because it appears on a reference diagram.

Where does the model actually run?

The location of the Foundry resource does not, on its own, tell you where inference happens. The deployment type does. Microsoft describes the processing scope of each, in its deployment types documentation:

Now apply that to South Africa North. At the time of writing, Microsoft's model availability tables show no chat model offered on regional Standard deployment there: only embedding models. There is no data zone for the region. Regional Provisioned lists a single, older model version that is scheduled to retire. In practice, a chat model used from a Foundry resource in South Africa North runs on a Global deployment, which means inference may happen outside South Africa. Our reference on AI model availability in South Africa goes through what "available" means in more detail; the short version is that a model appearing in the portal is not a processing-location commitment.

This is where Fynbos made its most consequential decision, and it is worth showing in full:

• All three products use a Global deployment for the chat model.
• Embeddings for retrieval use regional Standard in South Africa North, and the AI Search index, Storage and Cosmos DB stay in South Africa North, so the stored knowledge and conversation state remain in the country.
• For the two member products, the privacy office recorded a cross-border processing decision relying on section 72 of POPIA, on the basis of the contractual protections in Microsoft's data protection terms.
• Data minimisation narrows what crosses the border: member identifiers (member number and ID number) are replaced with a session token inside the application before any prompt leaves it.
• The decision is re-verified before production and at every model change.
• Fynbos considered Regional Provisioned and rejected it: one ageing model version and a fixed capacity cost, for a residency benefit the legal analysis did not require.

That last analysis was Fynbos's own lawyers' call, not the architects'. POPIA does not impose a blanket requirement that personal information stay in South Africa. Section 72 sets conditions for transfers abroad, including adequate protection through law, binding corporate rules or a binding agreement, consent, and contractual necessity; our plain-language piece on why POPIA does not require your data to stay here explains them. Health information is special personal information, which raises the bar for everything else in the processing. This article is not legal advice: the determination belongs to your legal and privacy advisers, for your organisation and your use case.

What the architecture must do is make the decision visible and checkable. Fynbos keeps a decision record for every model deployment:

model_deployment:
  product: "Member Desk copilot"
  model_and_version: "Record exact value"
  deployment_type: "Global Standard"
  azure_region: "South Africa North (resource)"
  processing_scope: "Any Azure region where the model is deployed"
  verified_on: "2026-10-09"
  cross_border_basis: "POPIA s72, recorded by the privacy office"
  data_classes_allowed: ["scheme rules", "benefit text", "tokenised member context"]
  prohibited_data: ["member number", "ID number", "free-text clinical notes"]
  fallback_model: "None (same model version only)"
  evaluation_pack_version: "Record version"
  business_owner: "Head of Member Services"
  technical_owner: "AI platform lead"
  review_date: "At every model change, and before production"

Common mistakes

• Calling a model "in-country" because the resource is in-country. The deployment type decides where inference may run. Record the model, version, deployment type, region and documented processing scope, and re-check before production.
• One API key for everything. It makes least privilege and investigation impossible. Use managed identities and pass user context.
• Presenting the gateway as the governance layer. It enforces shared runtime policy; it cannot judge correctness, retrieval permissions or business authority.
• Building on a preview tier without saying so. Name the preview features in the design and keep a production path that does not depend on them.
• Routing to any model that answers. A fallback must pass the same quality, language and processing-location requirements.
• Turning on semantic caching for member data. A shared cache is a shared data store.

Evidence a reviewer should ask for

• The identity chain for each product, showing which identity is carried at each hop.
• Role assignments by persona and scope, with no standing individual privileged access.
• The managed identity register.
• Gateway policies per product: model allow-list, token limits, metrics.
• A model deployment record for every deployment, with processing scope and the date verified.
• Where personal information may cross a border, the recorded legal basis and the minimisation applied.

What to take from this part

Identity establishes who may ask. Keep human, application, agent and tool identities distinct along the whole chain.

The gateway applies shared runtime policy. The product still decides what a request means and what action is allowed.

The deployment type, not the resource region, decides where inference runs. At the time of writing, a chat model used from South Africa North runs on a Global deployment.

Make the processing decision visible. A dated record, a legal basis from your advisers, and minimisation before the prompt leaves.

Next in the series: RAG is a data security system.

How CloudNala can help

We help organisations design the identity and access model around an AI platform: replacing shared keys with managed identities, defining personas and role scopes, configuring the AI gateway as a production control rather than a preview experiment, and producing the dated model deployment records that let privacy and legal teams make a real cross-border decision instead of approving a region dropdown.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Production Readiness Review or write to us at consult@cloudnala.co.za