Evidence an AI Patient Messaging Pilot Should Produce for the AI Governance Committee
A patient messaging pilot should give the committee six artifacts: the policy scope in force, PHI handling and redaction records, escalation and human-approval records, blocked or modified interactions, retention and access controls on the evidence itself, and pass/fail criteria set before launch. The strongest evidence is produced by policy enforcement in the execution path at the moment of each interaction. Evidence reconstructed afterward from application logs or review is weaker.
Treat the pilot as evidence production, not a performance trial
Most AI patient messaging pilots reach the AI Governance Committee with anecdotes, satisfaction scores and vendor assurances. Those inputs describe how the pilot felt. They do not show what the AI was permitted to do, how PHI was handled, when messages were escalated to clinical staff or blocked, or whether controls held under real traffic. A committee cannot approve, constrain or reject expansion on that basis, and the organization cannot defend the decision afterward.
The more useful framing is that the pilot exists to produce evidence. Each committee question should map to a specific artifact, and each artifact should be defined before the first message is drafted. The Security Rule requires covered entities and business associates to ensure the confidentiality, integrity and availability of the ePHI they create, receive, maintain or transmit. That is the scope against which pilot PHI handling will be judged.
The evidence covers individual interactions and tool calls. It does not provide transaction-level governance or end-to-end tracing of agentic workflows, and the committee should read the package with that scope in mind.
The six artifacts the evidence package should contain
Define each artifact in the pilot charter, name an owner for it, and decide how it will be produced. Each item below is expected at the level of individual interactions, not as a sample.
- 1. Policy scope in force: A written statement of what the AI may draft or send, and what requires clinician review. Every interaction record should carry the policy version applied at that moment, so the committee can tie any message to the rules in force.
- 2. PHI handling and redaction records: For each interaction, a record of what PHI appeared in prompts and outputs and what was redacted, with timestamps. A summary statement that PHI was protected is not equivalent.
- 3. Escalation and human-approval records: For each AI-drafted message, who reviewed it, who approved it and when. ASTP/ONC SAFER guidance says AI-generated recommendations should be reviewed and approved by humans who take responsibility for them before they reach patients.
- 4. Blocked and modified interactions: Each decision to block, redact or escalate, with the reason and policy version. Blocks and modifications show whether controls actually acted on real traffic.
- 5. Disclosure and awareness evidence: Proof that procedures making patients and clinicians aware that AI is in use were followed where possible, as SAFER guidance expects.
- 6. Retention and access controls on the evidence: Who can read, export or alter the evidence, how long it is kept and where it is stored. Evidence that cannot be protected is a liability of its own, since it contains PHI-adjacent content.
Runtime enforcement evidence versus reconstructed evidence
The committee should ask a plain question of every artifact: was this recorded because a control made a decision in the execution path before the message was sent, or was it inferred later from application logs and review? This is the author's analysis rather than a regulatory distinction, but it affects how much weight the evidence can bear.
Evidence produced by enforcement records the decision itself. Each interaction has an outcome (allow, redact, block, or escalate for human approval), a timestamp, the policy version applied, and the redaction or approval record that resulted. The record exists because the control acted, so it covers every interaction that passed through the path, including the ones nobody thought to review.
Reconstructed evidence starts from application logs or post-hoc review and works backward. It can show what was sent, but it often cannot show what the policy was at that moment, whether a decision was made before sending, or whether an interaction bypassed the logging layer. Gaps are hard to detect because the absence of a record looks the same as the absence of an event. Sampled review has the same weakness. SAFER guidance calls for ongoing local monitoring, and a sample cannot confirm that controls held across all pilot traffic.
Reconstruction is not worthless. Clinical review of message quality remains necessary. It should supplement enforcement records, not substitute for them.
Mapping the evidence to HIPAA and NIST AI RMF
Section 164.312 sets technical safeguards under the Security Rule, and paragraph (b) addresses audit controls. The pilot's per-interaction decision records, redaction records and approval records are the artifacts most directly relevant to that standard. Have the privacy officer and counsel confirm the artifact-to-requirement mapping against the current regulatory text and record it in the evidence package.
On regulatory status: HHS published a proposed rule to strengthen Security Rule cybersecurity of ePHI in the Federal Register in January 2025. As of November 2026, it should be treated as a proposal, and the current text governs. Design the pilot to the current rule and keep records in a form that can be extended if requirements change.
The NIST AI RMF offers a useful organizing lens through its four functions. Govern covers the policy scope and named ownership. Map covers the documented use case, its boundaries and its PHI exposure. Measure covers monitoring records and the blocked, modified and escalated interaction data. Manage covers escalation outcomes, incident records and the decision to constrain or expand. Using these functions as headings in the committee report makes gaps visible without implying a formal certification.
Set pass/fail criteria and accountability before launch
The committee should fix its criteria before the pilot begins, and they are locally defined. No regulation supplies thresholds for this purpose. Examples of criteria that make sense for a patient messaging pilot: every message sent to a patient has a matching approval record; every interaction record carries a policy version; no PHI handling event is missing a redaction or handling record; evidence coverage covers all pilot workflows, not a sample; and access to the evidence is limited to named roles. A criterion that cannot be checked from the evidence should be rewritten until it can.
Also decide in advance what each outcome triggers. Meeting the criteria may justify expansion with constraints. Missing evidence should count as a failure, not as an inconclusive result, because the pilot's purpose is to produce evidence the committee can rely on.
Accountability needs the same precision. SAFER guidance frames oversight as shared clinical, technical and administrative responsibility, and ASTP data on hospital predictive AI show that accountability for evaluation commonly sits with multiple entities. Those data cover predictive AI, not generative messaging, but the lesson transfers: the package should state who signs off on policy scope, PHI handling, clinical review and evidence retention. Monitoring should run throughout the pilot using local data, not only as an end-of-pilot survey.
Where runtime enforcement fits
Trussed enforces policy in the execution path for individual interactions and tool calls: allow, redact, block, rate limit, and escalate for human approval. Evidence is generated by that enforcement and mapped to HIPAA, including 164.312(b) audit controls, and to NIST. Trussed can be deployed self-managed in the customer cloud, with configurable retention and data residency, which bears on the sixth artifact above. It runs in production with healthcare organizations.
The checklist in this guide applies whichever tooling a pilot uses. If a vendor cannot show per-interaction decisions, policy versions and approval records produced at runtime, the committee should treat the evidence as reconstructed and weigh it accordingly.
Committee question and the artifact that answers it
What was the AI allowed to do?
Versioned policy scope in force for each interaction
How was PHI handled?
Per-interaction redaction and handling records
Who approved what reached patients?
Escalation and human-approval records
What was stopped?
Blocked and modified interaction records
Can the evidence be trusted?
Retention, access control and enforcement provenance
What counts as success?
Pass/fail criteria fixed before the pilot starts
Define the evidence package before the pilot starts
See how enforcement in the execution path produces per-interaction evidence for patient messaging pilots.
See How Evidence Is ProducedHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.