Deploying an AI Governance Proxy in Front of Azure OpenAI Private Endpoints

Network path and identity: the two controls

The pattern relies on two controls that work together. The network control makes the proxy the only path to the model. The identity control makes the proxy the only authorized caller on the Azure OpenAI resource.

The pattern separates callers, the proxy, and model access into distinct network and identity zones. Each zone has one permitted way in.

Reference topology

Request path with a governance proxy

1. Client or agentAuthenticates to the proxy with its own identity.
2. Governance proxyApplies policy, attributes the caller, and writes the audit record.
3. Private endpointReachable only from the proxy subnet.
4. Azure OpenAIPublic network access disabled.
5. Log platformReceives structured events for monitoring and SIEM use.

Zones and their configuration

Implementation sequence: making the proxy the only path

1. Disable public network access on the Azure OpenAI resource.
2. Place the private endpoint in its own subnet, and block client networks from that subnet.
3. Link the private DNS zone only to the network where the proxy runs, so the model hostname resolves to the private endpoint only for the proxy.
4. Grant access on the resource only to the proxy's managed identity.
5. Point applications, agents, and tool runtimes at the proxy's internal hostname, never at the Azure OpenAI hostname.

Confirm each step against the validation checklist at the end of this guide.

Identity, attribution, and policy enforcement points

Clients authenticate to the proxy as themselves. The proxy authenticates to Azure OpenAI with its own managed identity, and carries the original caller's identity into every policy decision and audit record.

Streaming, latency, availability, and logging

Run two or more proxy instances behind an internal load balancer in a dedicated subnet, and keep them stateless where possible so instances can be replaced without losing policy state. Taking one instance offline should not interrupt traffic or open a direct route.

Streamed responses should complete normally, and a policy violation mid-stream should behave as designed. The proxy emits structured events to a central log platform, from which they can be forwarded to a SIEM.

Validation checklist

• A direct call to the Azure OpenAI hostname from a client network fails at resolution or connectivity.
• A call with a valid key or token from outside the proxy is rejected.
• A request for a deployment not on the allowlist is denied and logged with the caller's identity.
• A request over the token or rate budget is throttled and produces an attributable record.
• A streamed response completes normally, and a policy violation mid-stream behaves as designed.
• Taking one proxy instance offline does not interrupt traffic or open a direct route, and every test above appears in the SIEM with its correlation ID.