Detecting whether clinicians send AI-drafted patient messages without meaningful review

Define meaningful review before measuring it

Oversight is an evidence problem. Because no single signal proves that a clinician reviewed an AI-drafted message, detection starts with what an organization can reasonably show: where a draft came from, what happened to it before it was sent, and whether human review quality holds up when sampled.

Defensible detection combines four sources of evidence:

• Draft provenance logging
• Draft-to-sent comparison
• Review interaction events
• Sampled quality audits

Review signals and their limits

Each signal contributes one link in an evidence chain for AI-drafted message oversight. None of them is sufficient alone, and each should be interpreted with its limits in mind.

Read signals together, and at the right level

These signals are read at the population level and paired with a runtime control that requires an explicit review action before an AI draft can be sent. Controls can vary by message category and risk level, with stricter rules for higher-risk categories.

Keep records defensible and minimize PHI

Records should be tamper-evident, exportable, and suitable for compliance, liability, or incident review. They should also capture edits, dwell time, and send actions without retaining unnecessary PHI, for example by storing a change summary or hash rather than full text where possible.

An audit and runtime-control architecture

1. Draft provenance record. At generation, log a draft ID, model and version, timestamp, message category, and the policy set in force.
2. Review event capture. Record structured events such as draft opened, edited, accepted, discarded, and sent, with timestamps.
3. Draft-to-sent linkage. Link the sent message to its originating draft and store a change summary or hash rather than full text where possible.
4. Runtime send gate. Enforce policy before release, for example requiring an explicit review action, with stricter rules for higher-risk categories.
5. Tamper-evident evidence store. Keep append-only, access-controlled logs that can be exported for compliance, liability, or incident review.
6. Audit sampling workflow. Draw stratified samples for human review and feed findings back into thresholds and policy.

Questions to ask when evaluating a governance platform

• Does it log draft provenance, including model and version, draft ID, and the link between the draft and the sent message?
• Can it capture edits, dwell time, and send actions without retaining unnecessary PHI?
• Can policy be enforced at runtime, such as requiring a review action before sending, rather than only reported afterward?
• Are records tamper-evident, exportable, and suitable for compliance, liability, or incident review?
• Does it support stratified sampling audits and discourage using review metrics as sole evidence about individual clinicians?
• Can it vary controls by message category and risk level?