Implementation Guide

    Deploy a Governance Proxy on GKE Inside VPC Service Controls for Vertex AI

    Run a self-managed, application-layer proxy on a private GKE cluster inside the perimeter. Make its service account the only identity that can call Vertex AI, route that traffic over a private path that stays inside the perimeter, evaluate every request against policy, and write each allow, redact, or block decision to storage you control.

    1. 1

      Client workloads

      Send model and tool traffic to the proxy and hold no direct Vertex AI permission.

    2. 2

      Proxy

      Runs in a dedicated namespace on a private GKE cluster and is the enforcement point.

    3. 3

      Policy decision point

      A separate service the proxy queries for each request.

    4. 4

      Private path to Vertex AI

      The only reachable route to Google APIs from the proxy subnet.

    5. 5

      Evidence store

      A bucket or equivalent in the customer project, inside the perimeter.

    6. 6

      1. Define the perimeter

      Place the cluster, evidence store, and Vertex AI projects in one perimeter.

    7. 7

      2. Establish the private path

      Configure Private Google Access with the restricted VIP or a Private Service Connect endpoint, plus DNS.

    8. 8

      3. Build the cluster and identities

      Private cluster, network policies, and federated service accounts.

    9. 9

      4. Deploy proxy and policy engine

      Run both in the isolated namespace with outbound limited to approved destinations.

    10. 10

      5. Configure policy

      Write rules that resolve to allow, redact, or block, and set fail-closed behavior.

    11. 11

      6. Route workloads

      Send all model and tool traffic through the proxy and remove direct permissions.

    Verify enforcement and evidence without perimeter violations

    • A direct call from a workload namespace to Vertex AI fails at both the network and IAM layers.
    • A call through the proxy succeeds and produces no perimeter violation entries.
    • A call from outside the perimeter is rejected.
    • Test requests that should be allowed, redacted, and blocked each produce one decision record.
    • With the policy engine stopped, the proxy behaves as designed (fail closed).
    • Evidence objects land in the intended region with the configured retention applied.

    Design the perimeter and private connectivity first

    Treat VPC Service Controls as a design constraint. Decide which projects hold the cluster, the evidence store, and the Vertex AI resources, and put them in the same perimeter. Cross-perimeter calls are where most unexpected violations come from, so keep the number of them small and explicit.

    For private connectivity there are two common patterns. Private Google Access with the restricted virtual IP and matching DNS is driven by routes and name resolution at the subnet level. It involves few objects, but the guarantee depends on DNS and routing staying correct. A Private Service Connect endpoint gives Google APIs a named internal address in your VPC, which you can address directly in firewall rules and monitor as a single object. It adds an endpoint to manage. The deciding question is which option lets you show that the proxy subnet has no other route to Google APIs or the internet. Whichever you choose, deny outbound traffic by default and allow only the approved destinations: the private API path, the policy engine, and the evidence store.

    For perimeter rules, name the proxy service account as the identity in any ingress or egress rule instead of relying on broad access levels. Add ingress rules only for callers that sit outside the perimeter, and egress rules only where the proxy must reach a resource in another project. Validate each rule in a non-production perimeter before enforcing it in production.

    Configure the cluster and identities

    Use a private cluster so nodes have no public addresses. Place the proxy and policy engine in their own namespace, ideally on a dedicated node pool, and apply a default-deny network policy. Then allow only three flows: client namespaces to the proxy, the proxy to the policy engine, and the proxy to the private API path and evidence store.

    Use Workload Identity Federation for GKE to bind the proxy's Kubernetes service account to a Google service account, so no key files exist in the cluster. Public research on Vertex AI agent deployments has shown how default permissions on Google-managed service accounts can be abused to reach credentials and internal resources. The mitigations apply here as well: scope permissions per deployment, use short-lived credentials, isolate execution contexts, and keep platform tokens out of containers.

    Keep the proxy identity separate from workload identities. Grant only the proxy the Vertex AI permission it needs. A workload that tries to skip the proxy then fails at the network layer and again at IAM.

    • Private cluster with no public node addresses
    • Dedicated namespace and node pool for the proxy
    • Default-deny network policy with three explicit allow flows
    • Workload Identity Federation for GKE, no exported keys
    • Minimal, per-deployment service account permissions

    Turn decisions into audit-ready evidence

    An enforced decision is only useful to an auditor if it is recorded in a form that survives. For each request, write a decision record with a timestamp, request identifier, caller identity, target model or tool, policy version, the decision, and which fields were redacted. Avoid storing raw sensitive payloads unless a requirement demands it, and if it does, apply the same classification and lifecycle rules as any regulated data.

    Write records to a bucket in the customer project, in a region chosen for residency, inside the perimeter, with a retention policy set by the customer. Carry the request identifier through so proxy decisions can be correlated with platform audit logs for the same project. Restrict write access to the proxy identity and read access to a small audit group.

    Trussed is one self-managed option for this layer. It is deployed in the customer cloud, enforces policy in the execution path with allow, redact, and block decisions, and generates audit-ready evidence from that enforcement, with configurable retention and data residency. The architecture above applies whether you build the proxy yourself or deploy a product.

    Request path inside the perimeter

    Workloads

    Applications and agents send model requests and tool calls to the proxy only.

    Governance proxy on GKE

    Enforces policy per request: allow, redact, or block.

    Private path

    Traffic to Vertex AI never leaves the perimeter or touches the public internet.

    Evidence store

    Decision records are written to customer-controlled storage with set retention and region.

    Review your perimeter design with an expert

    Walk through proxy placement, identity scoping, and evidence storage for your Vertex AI environment.

    Talk to an Expert
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.