Applying enterprise policy to third-party models in Vertex AI Model Garden
Apply policy to third-party models in three layers. Use cloud-level controls such as IAM and Organization Policy constraints to decide which models can be enabled and by whom. Use network and data perimeter controls such as VPC Service Controls to bound where traffic and data can flow. Then enforce policy on each prompt, response and tool call at runtime, in the execution path, because the first two layers do not decide what an individual interaction may contain.
Why one control layer is not enough
Model Garden gives teams access to partner models, open models and models they deploy themselves. A regulated enterprise did not build or host most of them, yet it still has to show that one consistent policy applied. The difficulty is that the available controls answer different questions.
Access-time and deployment-time controls answer whether a model may be used, by which identities, in which projects. Perimeter controls answer where requests and data may travel. Neither set decides whether a specific prompt contains data that should be redacted, whether a response should be blocked, or whether a tool call an agent proposes should proceed. Those are interaction-level questions, and they have to be answered while the interaction is happening.
NIST AI 600-1 points the same way. For third-party generative AI resources, it calls for applying organizational risk tolerances and controls, and it lists procurement processes next to filtering of inputs and outputs, grounding, fine tuning and retrieval-augmented generation. NIST AI 600-1 is voluntary guidance, not law, but it is a framework regulated enterprises commonly map their controls to. Treating access and interaction filtering as separate layers is consistent with it.
What each layer decides, and what it leaves open
Use this table to assign each requirement to the layer that can actually satisfy it. Where a requirement falls in the right-hand column of an earlier layer, it needs the next one.
- Cloud-level access and availability (IAM, Organization Policy constraints): Decides which models can be enabled and which identities and projects can reach them. Does not inspect prompt, response or tool-call content. Evidence: configuration state and administrative access records.
- Perimeter and data handling (VPC Service Controls, data-handling settings): Decides where traffic and data can flow and which resources sit inside a boundary. Does not decide whether content crossing an allowed path is acceptable. Evidence: perimeter and settings configuration.
- Endpoint and deployment settings (self-deployed open models): Governs how an endpoint you operate is exposed and who calls it. Does not apply content policy to individual requests by itself. Evidence: deployment configuration and change records.
- Runtime enforcement on interactions: Evaluates each prompt, response and tool call and applies allow, redact, block, rate limit or escalate. Evidence: a decision record for each interaction showing the policy evaluated and the action taken.
Partner models and self-deployed open models differ in who holds the responsibility
A partner model consumed as a managed API and an open model deployed to an endpoint you operate present different responsibility splits. With an API, the provider runs the model, and how data sent to it may be used is typically governed by the provider's policies and any business agreements. That makes contract review and data-handling terms part of your policy work, not only technical configuration. With a self-deployed endpoint, your team owns more of the serving surface: exposure, identities, network placement and what calls the endpoint.
Neither case removes the interaction-level gap. A managed API still receives whatever prompts your applications send, and a self-deployed endpoint still returns whatever the model generates. Record each model's source type and licence class in your inventory. Under the EU AI Act, as of November 2026 and as amended by Regulation (EU) 2026/1744, general-purpose AI model providers must keep technical documentation and supply information to providers of AI systems that integrate the model. The open-source exclusion has exceptions, including high-risk systems and systems under Articles 5 and 50, so do not assume open models fall outside scope. Collect and retain the documentation that comes with each third-party model, and take legal advice on whether your organization acts as provider or deployer for a given use case.
A sequence for implementation
Order matters. Each step narrows what the next one has to handle.
- 1. Inventory models and identities: List every model in use, its source type (partner API, open model, self-deployed), its licence class, and the service identities that call it. Update the acceptable use policy to cover proprietary and open-source generative AI and third-party personnel.
- 2. Restrict availability and access: Use Organization Policy constraints and IAM to limit which models can be enabled and which principals and projects can reach them. Default to deny for models not on an approved list.
- 3. Set the perimeter: Place relevant projects and resources inside a VPC Service Controls perimeter and review data-handling settings for each model class.
- 4. Reduce service identity scope: Review default permissions on service accounts and agent identities. Unit 42 research summarized by OWASP found an agent in Vertex AI Agent Engine that inherited excessive default permissions. That finding concerns agents, not Model Garden model access, but it shows why identity scoping needs its own review.
- 5. Enforce interaction policy at runtime: Put policy in the execution path for prompts, responses and tool calls, with actions to allow, redact, block, rate limit or escalate for human approval.
- 6. Define re-assessment triggers: Re-assess risk after fine-tuning, after adding retrieval-augmented generation, and when a third-party model is used for a use case not covered by initial testing.
Where runtime enforcement sits and what it covers
Runtime enforcement means the policy decision happens before a prompt reaches a model, before a response reaches a user, and before a tool call executes. Review after the fact can detect a problem, but it cannot prevent the disclosure or the action. For consequential use cases, enforcement in the path is the difference between a policy that was applied and one that was only written down.
The actions needed are few. Allow what complies. Redact sensitive data in prompts or responses. Block disallowed content or tool calls. Rate limit abnormal volumes. Escalate to a person when an action needs human approval. OWASP's mitigations for the agent permission issue include validating tool and identity usage through policy engines, which supports a separate runtime check on tool calls.
Trussed provides this layer. It enforces policy in the execution path across models, agents, tools and MCP servers, on individual interactions and tool calls, with allow, redact, block, rate limit and escalate for human approval. It runs 100+ models under one policy plane, which matters when a Model Garden estate mixes partner, open and self-deployed models and each would otherwise need its own rules. It can be deployed self-managed in the customer cloud, with configurable retention and data residency. Trussed operates alongside the native controls above and does not replace them.
Evidence: what each layer lets you demonstrate
Auditors ask two different questions. The first is whether the right controls were configured. Access, constraint, perimeter and endpoint settings, plus the records of changes to them, answer that. The second is whether policy was applied to a specific interaction. Platform records that show a model was invoked establish that a call occurred. They are not by themselves proof that a content policy was evaluated on that call and what action followed.
That second question needs a decision record per interaction: which policy was evaluated, what was found, and whether the result was allow, redact, block, rate limit or escalate. Trussed generates continuous audit-ready evidence as a by-product of enforcement, mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. In production it has generated 30+ GB of audit evidence per day, which indicates the volume a regulated enterprise should plan to retain and query. Whatever tooling you choose, set retention and residency requirements early, since evidence is only useful if it persists as long as the audit period requires.
Three policy layers for Model Garden models
Access and availability
IAM and Organization Policy constraints decide which models can be enabled and who can reach them.
Perimeter and data handling
VPC Service Controls and data-handling settings bound where traffic and data can go.
Runtime enforcement
Allow, redact, block, rate limit or escalate each prompt, response and tool call.
Close the interaction-level gap
Talk through how runtime enforcement and audit-ready evidence fit alongside your existing Google Cloud access and perimeter controls.
Talk to an ExpertHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.