How to Detect Wrong-Patient PHI in AI-Generated Notes and Patient Messages
Detect wrong-patient PHI by comparing every AI draft against the identity and scope of the patient it is meant for, before the draft is saved or sent. Apply three checks (identifier matching, context-scope validation, cross-patient entity comparison) at the retrieval, tool-call and output stages, map each finding to a response action (allow, redact, block, escalate), and retain evidence of each check and its outcome. Treat the result as a feed into human review, not a replacement for it.
Where to place checkpoints in the execution path
- 1
Retrieval stage
Verify that each retrieved document or record carries the active patient's identifier and encounter scope. Drop or flag anything outside it before it reaches the model.
- 2
Tool-call stage
Inspect each tool response for patient identifiers that differ from the active patient. A mismatch here is the earliest reliable signal that a query was mis-scoped.
- 3
Output stage
Compare the finished draft with the active patient's identifiers and known details. This is the last point before the note is saved or the message is sent.
Evidence to retain for audit controls and investigation
- Patient ID and responsible clinical user for the interaction
- AI version and timestamp
- Which checks ran at which stage, and each result
- Action taken (allow, redact, block, escalate) and the policy that triggered it
- Reviewer identity and decision for escalated items
- What was redacted or blocked, retained under access controls appropriate to PHI
Why this is a context-boundary problem
A drafting model has no inherent notion of which patient a request concerns. It writes from whatever context the surrounding system places in front of it. If that context includes another patient's data, the draft can reproduce it fluently and with no visible sign of error. That is why wrong-patient PHI is better handled as a context-boundary and output-validation problem than as a model-quality problem. A better model given the wrong context still produces the wrong note.
The following entry paths are engineering inferences about how context can cross patient boundaries, not documented findings. Retrieval can return a record for a similar name or a mistyped identifier. A session can carry context from a previous patient into the next request. A clinician working in several charts can leave the assistant bound to the wrong one. A tool call can return another patient's data when a query is under-specified. Each path puts a second patient's identifiers or clinical details into the prompt, and the model may blend them into one output.
This differs from a model reproducing personal information from its training data, which the HHS Security Rule proposal of January 2025 describes as a disclosure risk for generative AI. The controls here address the context your own systems assemble at request time. The consequence is similar: an impermissible disclosure when a message reaches the wrong patient, or a documentation error when a note lands in the wrong chart.
The three detection checks
Identifier matching compares names, dates of birth, record numbers and similar fields found in the draft or context against the active patient. Health IT already uses two rule-based approaches: exact field matching and matching of text strings by proportion of matching characters. Neither depends on training data, which makes them predictable to test. Exact matching is strict and will miss typos and formatting variants. Fuzzy matching tolerates those but raises the risk of treating two similar patients as the same person. Patients with similar names are at higher risk of wrong-patient error, so name-only matching is a weak point. Combine at least two independent identifiers where you can.
Context-scope validation asks whether the data in play belongs in this interaction at all. It checks that every retrieved item and tool result maps to the active patient and, where relevant, the active encounter. It does not need to read the draft, so it is cheap to run and well suited to the retrieval and tool-call stages.
Cross-patient entity comparison looks for more than one patient's identifying details in a single output: two names, two dates of birth, or a medication list attached to a different person than the salutation. It catches blending that identifier matching can miss when the correct patient's identifiers are also present. It depends on entity extraction from free text, so expect misses and false alarms, and tune it on your own data.
No check is complete. Missing identifiers, shared family names and legitimately multi-patient content such as a family history can all produce ambiguous results. Design for ambiguity by deciding in advance what each uncertain result should trigger.
Mapping findings to response actions
Which action fits which finding is a policy decision for your organization. The mapping below is a reasonable starting point to adapt per check and per stage. Treat same or similar names across patients as a higher-risk condition and lean toward escalation.
- Allow: All checks pass and identifiers are unambiguous. Record the checks and the result so the pass is itself evidence.
- Redact: A stray identifier appears in an otherwise correct draft, for example an incidental name in a patient message. Remove it and record what was removed. Redaction is a poor fit for blended clinical content, where removing a term can leave a misleading note.
- Block: Retrieved context or tool output belongs to a different patient, or the draft contains clear cross-patient clinical details. Stop the interaction before the note is saved or the message is sent.
- Escalate: The result is ambiguous: similar names, missing identifiers or a partial match. Route to a human reviewer with the finding attached. Human approval before AI-drafted content reaches patients is expected in ONC SAFER guidance.
Limits of detection and where runtime enforcement fits
Expect false positives, especially from fuzzy matching and entity extraction, and expect some misses. SAFER patient identification guidance is explicit that technology configuration alone cannot ensure accurate identification, and that staff need training and reliable procedures. Build the checks to narrow what humans must review and to stop clear errors, not to remove review. Measure false-positive rate and reviewer workload on your own data before setting thresholds, and test any redaction or detection tooling the same way.
The timing difference is the main reason to enforce in the execution path. Clinician spot checks and retrospective audits find errors after a note is in the chart or a message is in the patient's portal, when the disclosure has already happened. A check applied before save or send can stop the output.
Trussed enforces policy in the execution path on individual interactions and tool calls: allow, redact, block, or escalate for human approval, enforced at runtime, not reviewed after the fact. It generates audit-ready evidence from that enforcement, mapped to HIPAA, and can run self-managed in your cloud with configurable retention and data residency. It runs in production with healthcare organizations. Its scope is the individual interaction and tool call, so your patient context binding and the definition of what counts as a mismatch remain your design decisions.
Wrong-patient checks across the execution path
Retrieval
Confirm retrieved context belongs to the active patient.
Tool call
Inspect returned records before the model uses them.
Output
Compare identifiers and entities in the draft to the target patient.
Action and evidence
Allow, redact, block or escalate, and record what was checked.
Apply wrong-patient checks where the output is produced
Walk through your retrieval, tool-call and output stages with a Trussed expert and see how enforcement and audit-ready evidence work for AI-drafted clinical content.
Request a DemoHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.