On-Prem vs SaaS AI Control Plane for Health Systems: A CIO Decision Framework
No deployment model is correct for every health system. On-premises keeps PHI and evidence entirely inside your boundary but leaves you with the most operational work. SaaS reduces that work, but it puts a vendor in the PHI path unless data is redacted first, and it requires a business associate agreement when ePHI reaches the vendor. Self-managed deployment in your own cloud sits between the two: enforcement and evidence stay in your environment, and your team runs it. Choose by scoring each model on PHI exposure, data residency, evidence ownership, latency, operations, integration and regulatory mapping.
Questions to put to every vendor
- Will any ePHI reach vendor-operated systems, and will the vendor sign a BAA covering the enforcement layer and every subprocessor or hosting provider?
- Where are interaction and tool-call records stored, who controls retention and deletion, and can we export them to support 164.312(b) review?
- Can redaction or de-identification occur inside our boundary before data reaches any vendor-hosted component?
- How are access rights defined for AI agents and tools, and how is each decision (allow, redact, block, escalate) recorded for review?
- What are the documented latency and availability characteristics in the inline path, and does the layer fail open or fail closed?
- Which models, agents, tools, and MCP servers are covered by one policy set, and how is evidence mapped to HIPAA and NIST?
What the layer actually does, and why placement matters
In this framework, the runtime enforcement layer sits in the path of AI traffic and applies policy to individual interactions and tool calls. For each one it can allow, redact, block, rate limit, or escalate for human approval. This is the page's own working definition, and it is deliberately narrow: it describes enforcement at the moment of use, not model testing or after-the-fact review.
Because the layer sees prompts, responses, and tool-call arguments, it will often see PHI. That makes its location a security and compliance decision, not only an architecture preference. Where it runs determines whether PHI leaves your organizational boundary, who holds the records of what the AI was allowed to do, and who is responsible when the layer is slow or unavailable.
These questions are often settled by vendor preference. A structured comparison against fixed criteria is more defensible to a compliance committee and easier to repeat as the AI estate grows.
What HIPAA changes between hosting models
The audit controls standard at 45 CFR 164.312(b) requires hardware, software, or procedural mechanisms that record and examine activity in information systems that contain or use ePHI. It is listed as Required, not Addressable. The standard is technology-neutral and does not prescribe where those mechanisms run. Section 164.312 applies to a covered entity or business associate, so a vendor that qualifies as a business associate carries the obligation too.
HHS guidance says a covered entity may use a cloud service to store or process ePHI if it has a HIPAA-compliant business associate agreement with the provider that creates, receives, maintains, or transmits ePHI on its behalf. The agreement sets the permitted and required uses and disclosures. HHS also says a cloud provider is not a business associate if it receives and maintains only information de-identified under the Privacy Rule process.
The practical consequence: a SaaS enforcement layer that receives raw prompts containing ePHI needs a BAA, and its handling falls under the Security Rule. If redaction or de-identification happens inside your boundary before anything reaches the vendor, the analysis changes. On-premises and self-managed deployments keep the enforcement layer in your environment, but if a cloud provider hosts that infrastructure and handles ePHI, a BAA with that provider may still apply. Confirm with counsel.
A January 2025 proposed rule would revise the Security Rule. As of November 2026, treat it as proposed, not as current obligation.
Scoring the three models against six criteria
Score each model from 1 to 5 for your organization on every criterion, weight the criteria by your risk posture, and compare totals. The summaries below describe typical trade-offs, not guarantees. Individual products vary, so verify each claim with the vendor.
- PHI exposure and data residency: On-prem: PHI stays inside your facility and residency is fully yours. SaaS: PHI may reach vendor systems, so a BAA and the vendor's hosting locations matter. Self-managed cloud: PHI stays in your tenancy, and you choose the region.
- Audit evidence ownership and retention: On-prem and self-managed: you hold the records and set retention and deletion. SaaS: the vendor holds the primary store, so confirm export rights, retention controls, and how records support 164.312(b) review.
- Enforcement latency and availability: The layer is inline, so it adds a hop. On-prem and self-managed can sit close to your AI traffic, with availability engineering on your side. SaaS adds network distance and depends on the vendor's uptime. Ask for fail-open or fail-closed behavior in every case.
- Operational ownership: On-prem: you handle patching, scaling, and monitoring. Self-managed cloud: you still run it, but with elastic infrastructure. SaaS: the vendor operates it, and you manage the relationship and the contract.
- Integration across models, agents, tools, and MCP servers: Check that one policy set applies to every model, agent, tool call, and MCP server you use today and expect to add. Deployment model affects network reach to each: internal tools are easier to reach from on-prem or your cloud.
- Regulatory mapping: Ask how recorded evidence maps to HIPAA controls and to frameworks such as NIST. Under SaaS, also map the vendor's business associate obligations. Under on-prem and self-managed, the mapping and the operating controls are yours to attest.
Choosing by risk posture and infrastructure maturity
On-premises fits organizations with a strict boundary policy, existing data center capacity, and a team that already operates security infrastructure at that level. The cost is operational: your team patches, scales, and monitors an inline component whose failure affects AI workflows. If your AI workloads run in a public cloud, an on-prem layer may also add network distance that you have to design around.
SaaS suits organizations that want the fastest start and the lightest operational load, and whose legal and security teams are comfortable with a vendor in the path. It works best when sensitive content is redacted before it leaves your boundary, or when a BAA, subprocessor visibility, and evidence export are firmly in place. The trade-off is that availability and evidence custody depend on a third party.
Self-managed deployment in your own cloud suits organizations that already run production workloads there and want PHI, evidence, and residency decisions to stay within their own tenancy without building everything from scratch. You accept operational responsibility, though the infrastructure is familiar. Because the platform team can use existing identity, network, and storage controls, this model often matches mature cloud programs.
A workable rule: if you cannot staff the operations, do not choose a model that depends on it. If you cannot accept PHI crossing the boundary, do not choose a model that requires it.
Where a self-managed deployment fits, with one example
Trussed is one example of the self-managed model. It runs in the customer cloud, with configurable retention and data residency, and enforces policy in the execution path across models, agents, tools, and MCP servers: allow, redact, block, rate limit, and escalate for human approval, on individual interactions and tool calls. Enforcement happens at runtime, not after the fact, and it produces audit-ready evidence mapped to HIPAA and NIST.
For context on operating record, Trussed has run in continuous production in a regulated enterprise for more than 18 months, with 99.95% uptime on live production traffic and more than 100 models under one policy plane. It runs in production with healthcare organizations, among four regulated verticals. Evaluate it, and any alternative, against the same six criteria and the same vendor questions above.
Three places the enforcement layer can run
On-premises
Runs in your data center. Maximum boundary control, highest operational ownership.
Vendor-hosted SaaS
Runs in the vendor's environment. Lowest operational load, PHI may cross the boundary.
Self-managed in your cloud
Runs in your cloud tenancy. Evidence and residency stay with you, operations are shared with the software supplier's tooling.
Run your own deployment comparison
Talk through your PHI boundaries, evidence requirements and infrastructure maturity with an expert, and see how enforcement evidence is produced in a self-managed deployment.
Talk to an Expert