Deploying an AI Governance Proxy with Terraform and Helm

Where the proxy sits in the traffic path

Agents and LLM applications have no direct route to models or tools. Every model, tool, and MCP request passes through the governance proxy, which evaluates policy and records the decision. Decision records are sent to an audit log destination outside the cluster.

Enforced traffic path

1. Agents and LLM applicationsWorkloads with no direct route to models or tools.
2. Governance proxyEvaluates policy and records each model, tool, and MCP request.
3. Models, tools, and MCP serversReachable only through the proxy.
4. Audit log destinationReceives decision records outside the cluster.

Dividing responsibility between Terraform and Helm

Terraform owns the foundations that outlive any single release. Helm owns the proxy workload and the configuration that changes with each release. Keeping the two separate means a proxy release does not alter the network, identity, or egress foundations underneath it.

A staged deployment sequence

Deploy in the following order so that each stage rests on a verified foundation.

1. Provision the foundations with Terraform. Create the network, cluster, identity, secrets backend, certificates, and egress controls.
2. Release the proxy with Helm. Use versioned values and policy configuration.
3. Restrict agent egress. Model, tool, and MCP traffic can then only leave through the proxy.
4. Verify. Confirm the egress restriction and audit log capture, and repeat after every change.

Preventing bypass and protecting credentials

The proxy governs only the traffic that reaches it, so the egress restriction is what makes the deployment effective. Agent workloads must have no direct route to model providers, tools, or MCP servers; the only path out is through the proxy.

Credentials need the same discipline. Provider credentials should not exist in agent pod environments, images, or manifests. The secrets backend provisioned with Terraform is the foundation for keeping them out of agent workloads.

Validation, upgrades, and rollback

The proxy is released through Helm with versioned values, and the foundations beneath it are managed by Terraform. Verify the egress restriction and audit log capture after every change to either layer.

Post-change verification checklist

• A test pod in an agent namespace cannot reach a model provider endpoint directly.
• The same pod reaches an allowed destination through the proxy.
• A disallowed model or tool request through the proxy is blocked and recorded.
• An audit event appears in the external destination with caller identity, destination, and policy decision.
• No provider credentials exist in agent pod environments, images, or manifests.
• Losing a proxy replica leaves no unmonitored path and the proxy fails as designed.