Implementation Guide
Deploying an AI Governance Proxy in a VPC with No Public Internet Egress
To deploy an AI governance proxy in a VPC with no public internet egress, inventory every outbound dependency the proxy has, replace each with a private path or an in-boundary service, and make the proxy the only reachable route for agent traffic.
Start with the dependency inventory
Replacing each outbound dependency with a private path or an in-boundary service means the following:
- Private endpoints for model services.
- An internal registry for images.
- Locally distributed policy bundles.
- An internal identity provider and certificate authority.
- Audit logs stored inside the VPC.
| Dependency area | How it is handled |
|---|---|
| Model and tool traffic | Reached through private endpoints or internal services, never a public route. |
| Policy and artifacts | Delivered from internal registries and local policy distribution. |
| Identity and certificates | Issued and validated by services inside the boundary. |
| Audit and telemetry | Stored in-VPC, with any export handled as a separate, deliberate step. |
Private connectivity for models and tools
Reference topology
A workable layout separates agent workloads, the proxy tier, and the private services the proxy depends on. Names vary by cloud, but the roles stay the same.
- Agent subnets
- Proxy tier
- Private endpoint subnet
| Layer | Role |
|---|---|
| Agent subnets | Run agents and LLM-backed applications. Their only permitted destination for model and tool traffic is the proxy. |
| Proxy tier | Runs the governance proxy on redundant instances behind an internal load balancer. It is the only tier allowed to reach model endpoints and tool backends. |
| Private endpoint subnet | Holds interface endpoints or equivalent private connections to managed model services. |
| Private DNS | Resolves model and internal service names to private addresses so clients cannot fall through to public resolution. |
Keeping policy, identity, and audit inside the boundary
| Layer | Role |
|---|---|
| Internal services | Container registry, package mirror, policy store, certificate authority, and identity provider reachable only on private addresses. |
| Audit store | Log storage and retention inside the VPC, with write access from the proxy and read access limited to security and compliance roles. |
Rolling out and verifying enforcement
Verify the proxy is the only path
Test that agents cannot reach models or tools any other way. Use the checks below.
- Security groups on agent workloads allow outbound traffic only to the proxy's listener, and nothing else.
- Route tables have no internet gateway or NAT route for agent subnets, and egress deny rules are in place as a second layer.
- Private DNS resolves model and tool hostnames to private addresses, and agents cannot use external resolvers.
- Model endpoints and tool backends accept connections only from the proxy tier.
- From an agent workload, test direct calls to a model endpoint, a tool, and a public address. Each must fail, and only calls through the proxy must succeed.
- Repeat the tests after every network or policy change, and alert on any denied direct attempt.
Plan a no-egress deployment with your team
Walk through your dependency inventory, private connectivity options, and enforcement checks with a Trussed AI expert.
Talk to an Expert