Healthcare AI Audit Evidence

    Logging Which Chart Data an AI Used to Draft a Patient Message

    To log which chart data an AI used, write one evidence record per drafted message at each point where chart data moves: retrieval or tool call, prompt assembly, model response and clinician approval. Record data categories and record identifiers, what entered the prompt, what was redacted or blocked, the policy decision and the reviewer action. Capture it at runtime in the execution path, not by reconstructing it from final-text logs.

    Where chart data moves, and where to capture it

    1. 1

      Retrieval or tool call

      Log each call that reads the chart: the tool or API used, patient ID, the data category returned (medications, labs, problems, note type) and record identifiers.

    2. 2

      Prompt assembly

      Log which retrieved elements were included, which were redacted and which were dropped. This is the step most deployments cannot reconstruct.

    3. 3

      Model response

      Log the output reference, AI version and configuration, tied to the same message ID as the inputs.

    4. 4

      Clinician review and send

      Log the reviewer, action (approve, edit, reject), timestamp and the identifier of the sent message, so the draft links to what the patient received.

    Evaluation questions for your deployment

    • For one drafted message, can you list the categories and record identifiers retrieved, those placed in the prompt, and those redacted or blocked?
    • Does the record include AI version, timestamp, patient ID, responsible clinician and output?
    • Is evidence captured at runtime in the execution path, or reconstructed from application or model-provider logs?
    • Is the clinician approval, edit or rejection recorded and linked to the specific draft and the sent message?
    • Is PHI inside the evidence store minimized, access-controlled, retained on a defined schedule and kept in your cloud environment?
    • Are the logs reviewed on a schedule, with automation where practical, and tied to an inventory of deployed AI systems?

    What the record has to answer

    Most deployments store the final text of an AI-drafted portal reply or result explanation. That does not answer the question a clinician, compliance officer or auditor will ask: which chart elements informed this draft? Answering it requires four facts for each message: what was retrieved from the EHR, what was placed in the prompt, what was redacted or blocked on the way, and who reviewed the output.

    HIPAA audit controls sit in the technical safeguards at 45 CFR 164.312(b). The closest official reference points for record design come from ONC. Its audit logging test methods, built on ASTM E2147-18, expect actions to be logged with date, time and data category. ONC SAFER guidance recommends a transaction log of AI use with AI version, date and time, patient ID, responsible clinical user, input data and output. The ONC test methods apply to certified Health IT Modules, and SAFER guidance is advisory. Use both as design analogues for a drafting layer, not as direct mandates.

    The January 2025 HIPAA Security Rule proposal adds direction. It stresses regular review of audit logs and says ePHI in AI models and data maintained for covered functions is protected by the HIPAA Rules. It was published as a proposal, so as of November 2026 treat it as a signal of where expectations are heading, not as a settled obligation.

    Designing the evidence record schema

    Start with the SAFER transaction log fields, then extend them with retrieval, prompt-inclusion and policy fields. The extensions are design choices, and they are what make the record useful for minimum-necessary review and disclosure investigations.

    Store categories and pointers instead of values where you can. ONC accepts category-level logging: recording access to a medication list suffices for certification without recording the specific medication. Identifiers that point back to the chart let an investigator retrieve the original data under normal access controls, and they keep copied PHI out of the evidence store. Make logging default-on and synchronize timestamps to a network time source. If your stack uses FHIR, these fields can be mapped to AuditEvent and Provenance resources, but the schema should be driven by the questions above, not by the resource shape.

    • Message and actor identity: Message ID, patient ID, responsible clinical user, and NTP-synchronized timestamps for each step.
    • AI system identity: AI version and configuration in effect, tied to an inventory entry for the deployed system.
    • Retrieval entries: Per element: data category, record identifier, and the tool call or query that fetched it.
    • Prompt inclusion status: Per element: included, redacted or blocked, with the reason.
    • Policy decision: Allow, redact, block or escalate, with the policy identifier and version that produced it.
    • Output and review: Output reference, reviewer action, and the sent message ID.

    How runtime policy decisions shape the record

    When policy is enforced in the execution path, the evidence is a by-product of each decision instead of a separate logging effort. Each decision changes what the record shows. An allowed retrieval appears as retrieved and included. A redaction appears as an element and category with an inclusion status of redacted, without the redacted value. A block appears as a retrieval attempt with no corresponding prompt content, which is the evidence that a restriction held. An escalation appears as a pending state, followed by the reviewer's identity and outcome.

    This is the model Trussed uses. It enforces policy in the execution path, with allow, redact, block, rate limit and escalate for human approval, and the enforcement itself generates audit-ready evidence. Decisions are enforced at runtime, not reviewed after the fact. Trussed runs in production with healthcare and pharma organizations, and its evidence is mapped to HIPAA.

    Be precise about scope. This kind of evidence attests to individual interactions and tool calls. It does not by itself provide end-to-end tracing of an agentic workflow, and it says nothing about model or dataset lineage. Describe it to auditors in those terms.

    Keeping the evidence store from becoming a second risk

    An evidence store that holds prompts, outputs and identifiers is itself an ePHI system. ONC notes that audit stores and error queues, along with support dashboards, can persist messages containing PHI. Apply the same safeguards you apply to other ePHI systems.

    Minimize first: category and identifier pointers in place of values, with full prompt and output content stored only if a defined review need justifies it. Restrict access separately from the application that produced the records, and limit who can read prompt content compared with who can read metadata. Make records tamper-evident. Set retention deliberately with your compliance and legal teams, because the right period depends on your policies and the reviews you must support. Keep the store inside your own cloud boundary so residency decisions stay with you. Trussed supports self-managed deployment in the customer cloud with configurable retention and data residency.

    If a vendor holds any of this evidence, confirm it can be produced on request. Business associates must make internal practices, books and records on PHI use and disclosure available to the Secretary.

    One drafted message, one evidence record

    Retrieved

    Chart categories and record identifiers pulled by retrieval or tool calls.

    In the prompt

    Which retrieved elements were actually passed to the model.

    Redacted or blocked

    Elements removed or stopped before reaching the model.

    Policy decision

    Allow, redact, block or escalate, with the policy that applied.

    Output

    The draft, tied to the message ID and AI version.

    Reviewer action

    Who approved, edited or rejected the draft, and when.

    See what a per-message evidence record looks like

    Walk through how policy enforced in the execution path produces audit-ready evidence for AI-drafted patient messages.

    Request a Demo
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.