"We use private endpoints" is one of the most common sentences in an AI security review, and one of the least informative. It tells the reviewer that some services have private addresses. It does not say how users get in, how the application reaches the model and the data, where an agent's outbound calls go, how administrators reach the environment, whether names resolve to the right addresses, or what is allowed to leave.
Private Link is a component. A network architecture is a set of paths, each with an answer for protocol, identity, encryption, routing, inspection, logging and failure. This part walks through those paths for an enterprise AI workload, and ends with the evidence that shows they actually work.
Fynbos Health Administrators is fictional, and its systems and figures are illustrative. In Part 1, Fynbos, a South African medical scheme administrator, gave its two member-data products, the Member Desk copilot and the Claims Query agent, a Foundry resource of their own. This part designs their network.
Who owns which part of the network
Microsoft's baseline Foundry chat architecture in an Azure landing zone draws a clear line. The platform team owns the hub: the firewall, Bastion, private DNS zones and DNS resolution, and hybrid connectivity over ExpressRoute or VPN. It also provisions the workload's spoke virtual network and its route tables. The workload team deploys its resources into that spoke and owns the network security groups on its subnets.
That split is the right one, and it has a consequence that is easy to miss: several of the things most likely to go wrong (DNS zone links, routes, firewall rules) sit with a team that did not build the AI product. The design has to make those dependencies explicit, or the product team will discover them one failed request at a time.
If the networking basics are unfamiliar, Cloud Networking 101 covers private endpoints, NAT and outbound access from first principles, and its landing-zone networking part covers hub-and-spoke ownership.
Ingress: how a request enters
Ingress is the route a request takes into the workload. For an AI product it might be Application Gateway with a web application firewall, Front Door, API Management, an internal load balancer, Microsoft Teams, a contact-centre platform or a private corporate network.
For each entry point, the design should state:
- whether the user channel is public or private;
- where TLS terminates;
- where authentication happens;
- whether web application firewall rules apply;
- how denial-of-service protection and rate limiting work;
- how the original user's identity reaches the application.
Fynbos's answer for the member products is deliberately narrow. Contact-centre agents reach the Member Desk app only from the corporate network, through an internal Application Gateway with WAF. There is no public entry point at all. The Claims Query agent is not a separate channel: agents invoke it from inside the copilot, so it inherits the same ingress and the same signed-in user.
East-west: the paths inside the workload
East-west traffic is the internal path between the application and its dependencies: Foundry, AI Search, Storage, Cosmos DB, Key Vault and anything else the product uses. Private endpoints remove public data-plane exposure, but each service needs its own endpoint, its own DNS record and its own access control.
This is where a common assumption fails. Creating a private endpoint for the Foundry resource does not create private endpoints for AI Search, Storage or Cosmos DB. Microsoft's network isolation guidance is explicit that those are separate. A design that shows one private endpoint on the Foundry box and none on the data services has secured the front door and left the side doors to chance.
Fynbos's member spoke has five private endpoints: Foundry, AI Search, Key Vault, Storage and Cosmos DB. Each one is in the inventory, with its approval state, and each one has public network access disabled on the service itself.
Egress: where agents make the network interesting
Egress is everything that leaves. A traditional chatbot calls a model and a search index. An agent may also call SaaS platforms, corporate APIs, MCP servers, email, ticketing systems or websites, and some of those destinations are chosen at runtime by a model.
Ask, for every product:
- Is all internet-bound traffic routed through a firewall?
- Which destinations are allowed, by name, address or service tag?
- Can a tool redirect the agent to an unapproved endpoint?
- Which services cannot be force-tunnelled, and what compensates?
- Are payloads inspected, and what privacy impact does inspection create?
- Could an agent exfiltrate retrieved data through a tool that is allowed?
Fynbos routes all agent egress through the hub firewall, with an allow-list of exactly two destinations: the claims administration API, which runs in Fynbos's own Cape Town data centre and is reached over ExpressRoute, and the SaaS ticketing platform. Everything else is denied. The last question on the list still matters, though: the ticketing platform is allowed, so the Claims Query agent's tool contract (Part 5) has to control what goes into a ticket. A firewall decides where data may go; it cannot judge what the data is.
Microsoft's landing-zone reference also names components whose outbound traffic cannot be forced through the platform firewall. At the time of writing (October 2026), that includes Application Gateway and AI Search. Unsupported paths like these need a redesign, a compensating control or an explicitly approved exception. Fynbos records both as exceptions in its network design, with the compensating controls written next to them, rather than leaving a reviewer to find them.
DNS is part of the security boundary
Private endpoints only work if names resolve correctly. An application calling the normal service hostname must get the private IP address when it asks from inside the authorised network. Foundry and its typical dependencies use several private DNS zones, including:
privatelink.cognitiveservices.azure.comprivatelink.openai.azure.comprivatelink.services.ai.azure.comprivatelink.search.windows.netprivatelink.blob.core.windows.netprivatelink.documents.azure.com
The first three serve the Foundry resource itself; the others serve AI Search, Blob Storage and Cosmos DB. It gets harder when corporate DNS, Azure DNS Private Resolver, on-premises clients and several spokes are involved. A design can have every private endpoint it needs and still fail, because a request resolves to the public address or does not resolve at all. (DNS, routes and gateways explains resolution in general terms.)
Fynbos learned this in its pilot. The copilot was first deployed into a new spoke, and the privatelink.search.windows.net zone had never been linked to that spoke's virtual network. The on-premises conditional forwarder that testers in the office relied on was also missing. So the AI Search hostname resolved to its public address. Public network access on the search service was already disabled, so every request failed with a 403 instead of travelling over the internet.
It failed closed. That was luck, not design: had public access still been enabled, as it was on the original Policy Assistant, the same mistake would have worked silently over a public path. The fix was structural. Zone links are now part of the landing-zone code the cloud platform team owns, created with the spoke rather than afterwards, and the network acceptance evidence includes a resolution test from every network that should reach the service.
Treat DNS ownership, forwarding rules, zone links and record lifecycle as design decisions, not as tickets raised when something breaks.
Managed network, your own network, or public access?
Foundry gives you more than one way to isolate a workload's network:
| Option | Best fit | Benefits | Trade-offs |
|---|---|---|---|
| Managed virtual network | Teams wanting simpler isolation with fewer custom controls | Less network administration | Less flexibility; check feature support and current status |
| Bring your own virtual network | Enterprises integrating with an established landing zone | Control over routes, subnets, DNS and inspection | More design and operational responsibility |
| Public access with identity controls | Early, low-risk experiments on non-sensitive data | Fastest start | Unsuitable for most production and regulated workloads |
At the time of writing, bring-your-own virtual network integration for Foundry is generally available, although Microsoft notes that some networking experiences remain in preview. The managed virtual network is configured through the CLI, Bicep or Terraform rather than the portal. Check the current status of both against your region and the features you need before committing a design.
Fynbos uses its own virtual network for the member products, because its landing zone, hub firewall and ExpressRoute already exist and the agent's egress must go through them. Its Policy Assistant development sandbox, by contrast, runs with synthetic policy documents, constrained identities and no route to production systems. That is a perfectly valid use of a lighter option. Network isolation should follow the data and the threat model. Calling every environment "enterprise private" usually adds cost without clarifying risk.
Separate human, workload and operator access
Five different groups touch an AI environment, and they should not share a path:
- End users reach the product through an approved application channel.
- Workloads use managed identities to reach models, knowledge and tools.
- Developers and data scientists need controlled access to projects, traces, evaluations and test assets.
- Platform operators manage resources, policy, networking and deployments.
- Security operations consume diagnostic and security telemetry.
Do not make a jump box the universal answer. Decide whether developers connect through the corporate network, VPN, ExpressRoute, Azure Bastion, managed workstations or private build agents, and make sure the pattern supports emergency response as well as normal work. Fynbos's developers reach the member projects from managed workstations on the corporate network; deployments run from private build agents inside the landing zone; and there is a documented break-glass procedure for the night something has to be fixed and the usual path is down.
Common mistakes
- Treating "private endpoint" as a network design. It is one component on one path. Show ingress, east-west, egress, DNS and operator access.
- One private endpoint on the Foundry box. AI Search, Storage, Cosmos DB and Key Vault each need their own.
- Leaving DNS to whoever raises the ticket. Zone links and forwarders belong in the landing-zone code, with tests.
- Allow-listing a destination and assuming the data is safe. The firewall controls where; the tool contract controls what.
- Ignoring paths that cannot be force-tunnelled. Name them, and record the compensating control.
- Calling every environment enterprise private. A synthetic-data sandbox does not need the production network.
Evidence a reviewer should ask for
A diagram is a claim. A production review should ask for proof:
- Public network access status for every PaaS dependency.
- The private endpoint inventory and approval states.
- DNS resolution tests from each authorised network.
- A denied-access test from an unauthorised network.
- Effective routes and firewall logs for agent egress.
- The approved outbound destination list.
- The build and deployment path into the private environment.
- Administrator access and the break-glass procedure.
- Evidence that logs reach the required security workspace.
- Known platform limitations and approved exceptions.
What to take from this part
Private networking is a set of verified paths. If the design cannot show ingress, internal dependencies, egress, DNS and operator access, it is not yet a network architecture.
Every dependency needs its own private endpoint and DNS record. The Foundry endpoint does not cover the rest.
DNS failures are security failures. Fynbos's failed closed by luck; build the zone links and tests in.
Agent egress needs an allow-list and a tool contract. One controls where data goes, the other controls what goes.
Next in the series: Identity before intelligence: who may call the model, and on whose authority.
How CloudNala can help
We help organisations turn "we use private endpoints" into a network design a reviewer can verify: mapping every ingress, east-west and egress path for an AI workload, settling DNS ownership between platform and product teams, documenting the paths that cannot be force-tunnelled, and assembling the acceptance evidence before production rather than after the first incident.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Production Readiness Review or write to us at consult@cloudnala.co.za