What Counts as On-Prem for an AI Control Plane: Data Center, Private Cloud, or Customer VPC

Why "on-prem" is ambiguous

Security, architecture, compliance, and procurement teams often use "on-prem" to mean different things. For one team it means a physically owned data center. For another it means any private cloud. For a third it means a virtual private network the customer configures inside a public cloud. A vendor can truthfully use the word for any of these, which makes claims hard to compare.

NIST terminology shows why the word is not enough. NIST defines a private cloud as infrastructure provisioned for exclusive use by a single organization. It may be owned, managed, and operated by that organization, by a third party, or by some combination, and it may exist on or off premises. It can be implemented at the organization's premises or outsourced to a hosting company. "Private cloud" therefore does not tell you who operates it or where it sits.

A VPC is a different construct again. NIST treats a customer-configured private virtual network inside a cloud provider's native network as distinct from local networks on enterprise premises, such as data centers and branch offices. The customer configures the network, but the provider supplies the underlying infrastructure.

Separate where it runs from who controls it

The useful move is to treat location and control as two axes. Location answers where the workload runs. Control answers who decides what crosses the boundary, who can read the data, and who can change the system. A data center operated by a managed-services provider and a VPC operated by the customer's own platform team can both be "on-prem" under some definitions, yet they differ sharply in control.

NIST frames this through security perimeters. A perimeter is a barrier to access, and outsiders reach resources inside only where a boundary controller enforces a policy over that access. NIST also notes that deployment models change where consumer-controlled perimeters sit, and so how much control the consumer has over what it entrusts to a cloud. For infrastructure-style services, the consumer does not manage the underlying infrastructure and has limited control over select networking components such as host firewalls.

For an AI policy enforcement layer, the enforcement point is the boundary controller. If it sits inside the customer boundary, prompts and tool calls are evaluated before they leave. If it sits outside, that traffic crosses the boundary to be evaluated. Test control, data, evidence, and operations independently for each model.

Comparing the three deployment models

The comparison below uses four dimensions: network isolation, data residency, operational responsibility, and audit evidence custody. These are general architectural characteristics, not guarantees. Each must be verified for the specific deployment, particularly isolation, key custody, and egress controls.

• Physical data center: Network isolation: set by the customer's own network design and perimeter. Data residency: the physical location is known and customer-chosen. Operational responsibility: the customer, or a contractor it directs, runs hardware through application. Evidence custody: storage and keys can remain entirely under customer administration.
• Private cloud: Network isolation: exclusive use by one organization, but the isolation mechanism depends on the implementation. Data residency: depends on whether it is on-site or outsourced. Operational responsibility: may be the organization, a third party, or both, so confirm it per component. Evidence custody: follows whoever administers the storage and holds the keys.
• Customer VPC in a public cloud: Network isolation: a customer-configured private network inside the provider's network, with egress rules the customer sets. Data residency: depends on where the provider infrastructure hosting the VPC is located and on customer configuration. Operational responsibility: shared, since the provider runs the underlying infrastructure. Evidence custody: depends on storage placement, access controls, and encryption key ownership.

What must sit inside the customer boundary

An AI system is more than a model. It includes data pipelines, APIs, integrations, orchestration logic, user interfaces, and deployment infrastructure. Residency and custody questions therefore cover more than where the model is hosted. For an AI policy enforcement layer, three components decide whether a deployment is genuinely self-managed.

The first is the enforcement point in the execution path. It evaluates each interaction and tool call at runtime. If it runs outside the boundary, the content it inspects has already left.

The second is policy configuration. NIST's service-mesh model separates a control plane, which defines the policies governing access, from a data plane, which enforces them at runtime. Applying that split to AI is an architectural inference, but it is a useful one. A vendor-hosted control plane that manages customer-side enforcement is a hybrid, not a fully self-managed deployment, even if the enforcement point is local.

The third is evidence storage. Audit evidence and telemetry should be stored where the customer sets retention and residency, with keys the customer holds. Evidence stored in a vendor environment is custody by the vendor, whatever the label on the enforcement point.

Organizations that place sensitive or regulated data in a public cloud must account for how access is controlled and secured. NIST names access controls and encryption as the means. Residency and retention requirements themselves come from your own regulatory obligations, so map them to each component before accepting a model.

Example: self-managed enforcement in the customer cloud

Trussed offers a self-managed deployment in the customer cloud, with configurable retention and data residency. It enforces policy in the execution path across models, agents, tools, and MCP servers, with actions such as allow, redact, and block. That enforcement generates audit-ready evidence, which is kept inside the customer boundary under the customer's retention and residency settings.

This is one example of the pattern described above: enforcement at runtime, not reviewed after the fact, with evidence custody staying with the customer. Apply the same questions to any vendor, including this one, and record the answers per component before choosing between a data center, a private cloud, and a customer VPC.