Logging Cached LLM Responses for Auditor Traceability

The questions an auditor needs answered

When a response is served from a cache, no model call occurs. A standard request log may show a successful response with no sign that it was reused. The auditor then cannot tell whether the answer was generated for this request or for someone else, possibly under different rules.

Four questions define the evidence requirement. Which original interaction produced this response? Was the requester authorized to receive it at the moment it was served? Which policy was evaluated at serve time, as opposed to when the entry was created? Was the entry still valid under the retention and invalidation rules in force?

A logging design is adequate only if each question can be answered from records, without relying on engineers' recollection of how the cache behaves. Published OWASP guidance for RAG systems points the same way: cache hits should be logged with the same detail as fresh retrievals, and the requester's identity should be recorded on every request.

Exact-match and semantic caches leave different traceability gaps

In an exact-match cache, reuse is justified by a key derived from the request, typically a hash of normalized input and relevant parameters. The log can record the key and the matched entry, and an auditor can confirm the match is deterministic. The open questions are what went into the key. If user context, policy version or prompt template version is excluded from it, two requests that should have been treated differently can collide.

In a semantic cache, reuse is justified by a similarity score against a threshold. The match is a judgment, not an identity. The record therefore needs the score, the threshold in force, the identifier of the matched entry, and ideally a reference to the query that created that entry. Without these, an auditor cannot assess whether a near-match was acceptable, and cannot reproduce the decision if the threshold has since changed.

In both cases the record should state plainly which path was taken: exact hit, semantic hit, or fresh generation. That single field makes sampling and population counts possible.

Minimum evidence record for a cache hit

The fields below are a proposed minimum, not a mandated schema. Each is tied to one of the auditor's questions. Log identifiers and hashes rather than raw query or response text where content is sensitive, and keep the content itself in an access-controlled store referenced by ID. This resolves the tension between full-fidelity audit records and the guidance to keep sensitive query content out of logs.

• Event type and match basis: Exact hit, semantic hit or fresh generation. For exact hits, the cache key; for semantic hits, the similarity score, threshold and matched entry ID.
• Origin interaction reference: The ID of the interaction that generated the cached response, with its creation timestamp and the policy and prompt versions in effect then.
• Requester and request: Requester identity, access-control metadata, query identifier and serve timestamp.
• Serve-time policy decision: The decision (for example allow, redact or block), the policy version evaluated at serve time and the basis for it.
• Retention and invalidation state: Entry expiry, whether it was invalidated, the reason and the retention rule governing the entry and its source material.
• Integrity marker: A record ID, content hash and ordering information so that alteration or deletion of records can be detected.

Risks to trace: stale reuse, cross-user leakage and policy drift

OWASP names three risks for cached responses in RAG systems: cross-user data leakage, stale permission enforcement and persistent poisoning. Its guidance is that a response cached for one user should not be served to another unless both have identical access rights, and that highly sensitive data should not be cached at all. Cached responses should also not be served after source-document permissions are revoked or changed, and a stale or potentially compromised entry should not be used as a fallback.

These map directly to record fields. Cross-user reuse is testable only if the origin requester's entitlements and the serving requester's entitlements are both recoverable. Staleness and policy drift are testable only if the record carries the policy and prompt versions at creation alongside the policy decision at serve time. Poisoning response depends on being able to list every requester who received an affected entry.

Invalidation must itself leave evidence. When permissions change, a prompt is revised or a source is found tainted, the log should show which entries were invalidated, when and why, so an auditor can confirm no hit was served afterwards.

Retention, integrity and framework mapping

Cached responses, embeddings and audit records follow different retention logic. OWASP advises that cached responses and embeddings not outlive the source document's retention period, while audit logs must stay consistent with legal retention and erasure requirements. Deletion processes should be checked to confirm that cached embeddings and prompt logs are actually removed, while retained evidence follows its defined rule. For integrity, common design options include append-only storage, hash chaining of records and restricted write access. The auditor's concern is whether a missing or altered record would be detectable.

On the regulatory side, the EU AI Act (Regulation 2024/1689) requires logging capabilities in high-risk AI systems that enable recording of events relevant to traceability appropriate to the system's intended purpose. Whether a given deployment is high-risk, and whether cache hits count as relevant events, is a classification and design decision the organization must document. As of October 2026, confirm the current consolidated text before citing specific obligations.

Trussed enforces policy in the execution path on each interaction, including those served from cache, so the policy decision is made at runtime rather than reviewed after the fact. The resulting audit-ready evidence is mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. Retention is configurable, and deployment is self-managed in the customer cloud. Its scope is individual interactions and tool calls.