AI Governance Deployment in a Health System: Data Center vs Your Own Cloud Tenant
Place runtime policy enforcement and audit evidence capture where they can see all PHI-bearing AI traffic in the execution path, and where the health system can protect, retain, and produce the resulting records. Both a data center and a customer-owned cloud tenant can meet HIPAA expectations. HHS permits any cloud configuration with a business associate agreement. The deciding factors are the PHI boundary, the network path to models and tools, evidence durability, residency, availability, and who operates the system. Hybrid patterns are common and should be designed deliberately.
Map the PHI boundary and the network path first
- 1
Clients and agents
Identify where prompts originate and whether agents act with their own credentials. 164.312(a)(1) names software programs as well as persons.
- 2
Models
Note whether each model runs in the data center, in the tenant, or in an external service. Externally hosted models raise the contract and risk analysis questions.
- 3
Tools and MCP servers
Tool calls can reach systems of record directly. Enforcement has to see these calls, not only the model request.
- 4
Enforcement point
Confirm it is inline for every route and decide what happens to requests if it is unavailable.
- 5
Evidence store
Record where prompts, outputs, model configuration, and approvals are written and who can read them.
Criteria for selecting a deployment model
- Which components in the execution path create, receive, maintain, or transmit ePHI, and is the enforcement and evidence layer one of them?
- Can the enforcement point see and act on traffic to every model, agent, tool, and MCP server, including those in other environments?
- What happens to requests if enforcement is unavailable, and is that behavior acceptable for clinical workflows?
- Where are evidence records stored, who can read them, and how are they backed up, restored, and produced for an auditor?
- Who holds contractual responsibility for availability, backup, recovery, and data return, and what duties remain with the health system?
- What staffing is required to operate enforcement and evidence capture across many models, and can it scale beyond the first use case?
Treat deployment location as a function of traffic, not hosting preference
Teams often frame this decision as cloud versus on-premises. That framing misses the point. Policy enforcement only works if it sits in the path that AI requests and tool calls actually travel. Audit evidence only helps if it is produced by that same enforcement point and stored where the health system can protect and retrieve it. Location follows from those two requirements.
The regulatory frame is the same in either environment. As of November 2026, 45 CFR 164.312 is in force. It requires audit controls that record and examine activity in information systems that contain or use ePHI (164.312(b)), access limited to persons or software programs granted access rights (164.312(a)(1)), and transmission security against unauthorized access to ePHI in transit. HHS has also stated that ePHI in AI training data, prediction models, and algorithm data maintained for covered functions is protected by the HIPAA Rules. The December 2024 Security Rule NPRM is a proposal only, and current requirements are not superseded on that basis.
A practical consequence follows. If the enforcement layer handles prompts or outputs containing PHI, or stores records of them, it is plausibly an information system that contains or uses ePHI. It then needs the same access control, encryption, backup, and activity review as any other system in scope. Validate that reading with your compliance team.
Data center versus customer-owned cloud tenant
The comparison below is an analytical starting point. Weight each criterion against your own traffic map, risk analysis, and staffing rather than treating either column as the default.
- PHI boundary: Data center: PHI stays inside the existing perimeter if models and tools are local. Cloud tenant: PHI enters a tenant you own, which involves a cloud provider and a BAA decision for that provider's role.
- Network path to models and tools: Data center: short path to local systems, longer path to cloud-hosted or external models. Cloud tenant: short path to models and tools in the tenant, longer path to on-premises systems of record.
- Evidence durability and retention: Data center: you design storage, backup, and off-site copies. Cloud tenant: durable storage services are available, but backup, restore, and data return still need contract and design attention.
- Data residency: Data center: location is fixed and known. Cloud tenant: location depends on the regions you select and restrict.
- Availability: Data center: capacity planning and failover are yours. Cloud tenant: shared responsibility with the provider, set out in the BAA and risk analysis.
- Staffing and operational ownership: Data center: infrastructure, patching, and storage operations sit with your teams. Cloud tenant: less hardware work, but cloud security, identity, and cost of operations still need named owners.
Evidence retention, residency, and recovery
Audit evidence for AI is high volume and per-request. It includes what was asked, what policy applied, what action was taken (allow, redact, block, or escalate), and any human approval. Size the store for that volume. For reference, a regulated production deployment can generate more than 30 GB of audit evidence per day, so retention settings have a direct storage consequence.
HIPAA treats activity review as a procedure, not just collection. Information system activity review is a required implementation specification, so the evidence store needs a recurring review process and a way to retrieve records for an auditor. Contingency planning applies as well: the Security Rule requires a data backup plan and emergency access procedure, and lists data backup and storage and applications and data criticality analysis as addressable. ONC SAFER guidance for EHR resilience points to encrypted off-site backups, regularly tested restores, and backup media separated from operational data. Extending that to an AI evidence store is a reasonable inference, not a stated requirement.
HHS notes that the cloud configuration may affect the risk analysis and any BAA terms, including availability, backup and recovery, and how data is returned after service termination. Define retention periods and residency constraints with counsel, then confirm the chosen environment can enforce them by configuration and demonstrate them on request.
Hybrid patterns and what self-managed deployment involves
Many health systems will run models in more than one place: a data center for sensitive workloads, a cloud tenant for scale, and an external service for specific capabilities. In that case, avoid one enforcement point per environment with inconsistent rules. Prefer a single policy definition applied wherever traffic passes, with evidence written to a store whose location and retention you control. Treat paths to externally hosted models as open design questions to validate with traffic tests, since coverage and latency depend on your network.
A self-managed deployment means the enforcement and evidence components run in infrastructure the health system owns, under its identity, network, and key management controls. The health system operates upgrades, capacity, monitoring, and recovery, and in return keeps PHI and evidence inside its own boundary.
Trussed supports this model as self-managed deployment in the customer's own cloud, with configurable retention and data residency. It enforces policy in the execution path across models, agents, tools, and MCP servers, and generates audit-ready evidence mapped to HIPAA. It runs in production with healthcare organizations, and has operated for more than 18 months in a regulated enterprise with 99.95% uptime on live production traffic and more than 100 models under one policy plane. For next steps, build the traffic map, pick the environment that minimizes PHI movement before enforcement, and test an evidence restore before go-live.
Where enforcement and evidence sit in the AI execution path
AI clients and agents
Clinical, operational, and administrative applications that send prompts and tool calls.
Enforcement point
Applies allow, redact, block, or escalate for human approval to each interaction.
Models, tools, and MCP servers
Hosted in the data center, the cloud tenant, or an external service.
Evidence store
Holds per-request records that may themselves contain ePHI.
Work through your deployment decision with an architect
Review your AI traffic paths, PHI boundary, and evidence requirements, and see how self-managed enforcement in your own cloud produces audit-ready evidence.
Talk to an Expert