Implementation Guide | Healthcare

    Logging LlamaIndex Retrievals as HIPAA Audit Evidence Without Storing PHI in the Log

    Record each LlamaIndex retrieval as a reference-based event: who made the request, which document and node identifiers were returned, which policy version applied, what decision it produced, and when. Capture the event at the retriever, node postprocessor, or query engine boundary, and exclude query text, chunk text, and response text before anything is persisted. Protect the store so alteration is detectable, then test the records against what an auditor needs to see.

    A PHI-free retrieval record schema

    1. 1

      Actor context

      User or service identity, role or group, application name, and a request or session ID used for correlation.

    2. 2

      Action and timestamp

      Action type (retrieval), plus a timestamp from a synchronized time source. ONC references NTP for date and time.

    3. 3

      Source references

      Opaque document ID, node ID, index or store ID, and a reference that resolves to the source under access control.

    4. 4

      Integrity reference

      A document or node version ID, or a keyed hash of the content version, so a reviewer can confirm which version was served.

    5. 5

      Policy decision

      Policy ID and version, decision (allow, redact, or block), and a reason code.

    6. 6

      Outcome

      Nodes requested, returned, and removed, retrieval scores if useful, and an error class or code in place of any error message.

    Storage, tamper evidence, and checking that the records work

    • Write audit records to a store the application cannot modify or delete, with alteration detectable by hash chaining or signing.
    • Record audit status changes, and restrict who can disable auditing.
    • Seed synthetic PHI into queries, documents, and forced errors, then search every log, trace, and exception store for it.
    • Hand an auditor one record and confirm it answers who, what, which policy, what result, and when without any content.
    • Confirm references resolve to the source only for authorized reviewers.
    • Schedule regular review of the records and document who performs it.

    What the audit control asks for, and what the record is for

    The text of 45 CFR 164.312(b) is short: implement hardware, software, and/or procedural mechanisms that record and examine activity in information systems that contain or use electronic protected health information. It is listed as Required. A separate Required specification, information system activity review, calls for procedures to regularly review records of system activity such as audit logs and access reports.

    The rule is technology-neutral, so the record format is your decision. That is useful for retrieval-augmented generation. The record has to show that activity occurred and let a reviewer examine it. It does not have to reproduce the content that was touched.

    ONC's health IT certification criteria offer a helpful analogy, with one caveat: they apply to certified health IT modules, not directly to a covered entity's RAG system. ONC states that a log showing a user accessed a patient's medication list is sufficient, and the log does not also need the specific medication. It also notes that a log may link to an original document instead of embedding it. Treat the audit record the same way: evidence of what was retrieved and what policy applied, with a reference back to the source.

    There is also a design reason to stay PHI-free. A log that holds query text or chunk text is itself a system containing ePHI, so it inherits the same controls you are trying to evidence. You would be securing, retaining, and producing a second copy of the data.

    Where to capture it in the LlamaIndex pipeline

    Capture at the points where you can see identifiers and decisions without handling content you then have to discard. Each point trades coverage against exposure, so choose deliberately.

    • Retriever: Sees the nodes returned for a request. Emit node and document IDs and scores here. Do not pass node text or metadata values through to the audit writer.
    • Node postprocessor: Runs after retrieval and is a natural place to apply an access or redaction decision, then record which nodes were kept, modified, or removed and why.
    • Callback or instrumentation events: Useful for timing and correlation across steps. Event payloads can carry query text, node content, and responses, so allowlist the fields you emit rather than forwarding whole payloads.
    • Query engine: The outermost boundary for a request. Bind actor identity and request ID here and write one summary record per request, including the final outcome.

    Keeping PHI out before anything is persisted

    The reliable pattern is an allowlist. Build the audit record from named fields only, and make the writer reject anything else. Scrubbing a full payload after the fact depends on recognizing every PHI pattern, and a miss means PHI is already in the store.

    The usual leak paths are the ones nobody designed as audit records. Review each one in your own configuration: callback or instrumentation handlers that serialize query, node, or response payloads; debug and verbose tracing left on in production; trace exporters that ship spans to external observability tools; and exception handling that includes input text in the message. For exceptions, record the class and a code, not the message string.

    Identifiers and hashes need the same scrutiny. Source URIs and file names can contain patient names or record numbers, and metadata fields can carry values that identify a person. Use opaque IDs and keep the mapping to real names in the document store, behind its own access controls. A plain hash of short or predictable content can be confirmed by guessing the input, so it is not a safe stand-in for the text. Use a keyed hash with the key held outside the audit store, or record a version ID instead.

    For the query itself, record a request ID and, if needed, a coarse category assigned by your own classifier. Do not store the string.

    Where runtime policy enforcement fits

    The pattern above can be built in application code, and it works best when the policy decision and the record come from the same place. If the audit record is reconstructed later from application logs, it depends on logs that may already contain PHI. If the decision is made in the execution path, the record is a by-product of the decision.

    That is the approach Trussed takes. Trussed enforces policy in the execution path, with allow, redact, and block outcomes applied to individual interactions and tool calls at runtime, not reviewed after the fact. That enforcement generates audit-ready evidence mapped to HIPAA, so the decision and the evidence share one origin. Trussed runs in production with healthcare organizations and supports self-managed deployment in the customer cloud, with configurable retention and data residency, which keeps the audit store under your control.

    The scope is individual interactions and tool calls. It does not replace your document-level access controls or your review procedures, and the record schema and verification steps above apply whichever way you implement them.

    PHI-free retrieval audit record

    Actor

    User or service identity and request ID, not the query

    Sources

    Document and node identifiers, not retrieved text

    Policy

    Policy version and decision: allow, redact, or block

    Time and outcome

    Synchronized timestamp, counts, and error class

    Review your retrieval audit design with us

    Walk through your LlamaIndex capture points, record schema, and evidence requirements with a Trussed expert.

    Talk to an Expert
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.