Health System AI Telemetry Data Residency: On-Prem Logs vs Vendor Cloud
AI telemetry is a data class separate from model inference data, and much of it can contain PHI. Prompts, outputs and tool-call records are the most likely to. Where that telemetry is written, who can access it and how long it persists should be set by requirements, not left to vendor defaults. A vendor cloud requires a business associate agreement and moves custody outside your boundary. On-prem gives the most control and the most operational work. Self-managed deployment in your own cloud sits between them. Redacting PHI before telemetry is written changes the decision, because it limits what any store holds.
Requirements to draft for vendor evaluation
- Telemetry inventory: a vendor-confirmed list of every telemetry type written, with the fields each contains.
- BAA coverage: a signed agreement that explicitly covers all telemetry types that may contain ePHI, in place before production data flows.
- Residency: named storage geography for primary stores, backups and sub-processors, with a statement on any storage outside the United States.
- Retention and deletion: customer-configurable retention by telemetry type, with deletion or return of data at termination and defined backup and recovery handling.
- Redaction: PHI redacted before telemetry is written, with the redaction rules visible and testable by your team.
- Access and evidence: who holds administrative access to the stores, how access is logged, and what audit evidence is retained for compliance review.
Telemetry is a data class, not incidental exhaust
Many AI evaluations focus on what the model sees at inference time and treat the records around it as operational detail. Telemetry deserves its own treatment. It persists after the interaction ends, it is often copied into monitoring and support systems, and it is read by people who would never see the original clinical workflow.
In practice, a health system produces five kinds. Prompts carry whatever the user or upstream application supplied, which in clinical settings can include names, identifiers and free-text notes. Completions can restate that content, and generative models can also reproduce personal information from their training data. Tool-call payloads hold the arguments and results of calls to EHR queries, search tools or code tools, and often contain structured patient data. Policy decisions record what was allowed, redacted or blocked, and may quote the content that triggered the decision. Metadata such as user, timestamp, model and endpoint is usually the lowest risk, but it can identify a patient when combined with other fields.
This classification is practitioner judgment, not a regulatory taxonomy. Treat it as an inventory exercise: for each telemetry type in each AI system, record whether it can contain ePHI, in what form, and who currently reads it. The HIPAA Security Rule applies to electronic PHI, and HHS's 2025 Security Rule preamble states that ePHI in AI training data, prediction models and algorithm data held by a regulated entity for covered functions is protected. Telemetry that holds ePHI should be handled on the same footing.
What HIPAA means for where telemetry is stored
HHS states that a covered entity or business associate may use a cloud service provider to store or process ePHI if it enters into a HIPAA-compliant business associate agreement and otherwise complies with the HIPAA Rules. The reverse also holds: using a cloud provider to maintain ePHI without that agreement is a violation. The provider may have a 30-day cure defense, but that protects the provider, not the health system.
For telemetry, the practical consequence is that a vendor-hosted store containing prompts or tool-call payloads makes the vendor a party that maintains ePHI. The agreement needs to cover every telemetry type that can contain ePHI, not just the primary service data. HHS also says that while a provider maintains ePHI, it may not use or disclose it inconsistently with the Rules, so contracts should state whether telemetry may be used for anything beyond service delivery.
Geography matters as well. HIPAA permits storage on servers outside the United States if an agreement is in place and the Rules are otherwise met, but HHS notes that overseas outsourcing may increase risk and raise questions about enforceability. Residency requirements should therefore name storage location, including backups and sub-processors.
The technical safeguards in 45 CFR 164.312, including the audit controls standard at 164.312(b), apply to both covered entities and business associates. Contracts should assign who is responsible for audit controls over each telemetry store, and who can show the evidence.
Comparing the three storage models
The comparison below uses six criteria. It describes the general tendencies of each model, and individual implementations vary. Redaction before write is a separate variable that can be applied in any model, and the next section covers it.
- PHI exposure: On-prem: PHI stays inside systems you operate. Vendor cloud: PHI sits with a third party that needs a BAA and carries its own sub-processors. Customer cloud: PHI stays in your tenant, with fewer parties maintaining it.
- Audit controls (164.312(b)): On-prem: you build and own the controls. Vendor cloud: responsibility is split by contract and depends on what the vendor exposes. Customer cloud: you hold the controls and the evidence in your own environment.
- Retention configurability: On-prem: fully yours, bounded by your storage planning. Vendor cloud: often fixed tiers or defaults, so confirm what you can set. Customer cloud: set by your policy, subject to the product's configurability.
- Access boundaries: On-prem: your identity and network controls apply. Vendor cloud: vendor staff may hold administrative access. Customer cloud: your identity and key management govern access.
- Evidence durability: On-prem: depends on your backup and immutability practices. Vendor cloud: depends on vendor retention and what is returned at termination. Customer cloud: durable under your own lifecycle rules.
- Operational burden: On-prem: highest, covering capacity, patching and recovery. Vendor cloud: lowest for you. Customer cloud: moderate, since you manage the deployment but not physical infrastructure.
Why pre-write redaction changes the residency decision
Residency decisions usually assume telemetry arrives at the store as it was produced. If PHI is redacted before the record is written, the store holds less, and the question changes from where to put PHI to whether PHI should be there at all. This is a design inference rather than a regulatory finding, but the reasoning is direct: data that is never written cannot be over-retained, over-shared or exposed through a misconfigured store.
This only works if redaction happens in the execution path, at the point where the interaction or tool call is evaluated, rather than in a cleanup job after storage. A post-hoc scrub leaves a window in which PHI exists in the store, in backups and in any downstream copies. Enforcement in the path also produces the policy decision record at the moment of the decision, which is the evidence compliance teams later need.
There are tradeoffs. Aggressive redaction can remove detail that incident responders want, so define what must be retained for investigation and where it is held under tighter access. Redaction rules need testing against real clinical text, since identifiers appear in free-form ways. And redaction does not remove the need for a BAA or a residency requirement. Telemetry can still carry indirect identifiers, and the vendor may still maintain other ePHI. It reduces exposure; it does not remove the obligation to decide where records live.
Where Trussed fits
Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants. Enforcement can allow, redact, block, rate limit or escalate an interaction or tool call for human approval, and it is enforced at runtime, not reviewed after the fact. That makes pre-write redaction of PHI a policy decision made before telemetry exists.
Trussed supports self-managed deployment in the customer cloud, with configurable retention and data residency, so the health system keeps evidence in an environment it controls. Enforcement generates audit-ready evidence mapped to HIPAA, among other frameworks. Trussed runs in production with healthcare organizations. Its scope is individual interactions and tool calls, and it does not provide end-to-end tracing of agentic workflows or model lineage.
Three places AI telemetry can live
On-prem logs
Written to infrastructure the health system operates directly.
Vendor cloud
Written to a store the AI vendor hosts and administers.
Self-managed in customer cloud
Written to a store inside the health system's own cloud environment.
Review your telemetry residency requirements
Talk through your telemetry inventory, storage model and redaction requirements with a Trussed expert.
Talk to an Expert