Logging Failed, Timed-Out, and Partial LLM Requests for Audit

Why non-success outcomes belong in the audit trail

Most LLM audit trails record successful request and response pairs. Failed calls, timeouts, interrupted streams, and truncated outputs are often dropped or written inconsistently as error-handling noise. For a regulated enterprise this leaves gaps in the record of what the system was asked to do, which policy applied, what was returned or withheld, and what the user or downstream system actually received.

Regulatory text points the same way. As of October 2026, the EU AI Act states that high-risk AI systems shall technically allow for the automatic recording of events over the lifetime of the system, and that logging should support traceability appropriate to the system's intended purpose. Whether a given use case is high-risk depends on its classification, so scope the record against intended purpose. A failed or partial interaction is still an event in the lifetime of the system.

An audit log, as commonly defined, is a chronological record that allows the original sequence of attributable transactions to be reconstructed, reviewed, and examined. That definition does not exclude transactions that failed. If the sequence has gaps wherever something went wrong, it cannot be independently verified.

Define the outcome states first

Start with a closed list of outcome states and require every request to end in exactly one. The taxonomy below is a working proposal. Each provider signals these conditions differently, so map provider-specific status codes, finish reasons, and stream events to your own states in one place, and record the raw signal alongside the normalized state.

• Provider error: The upstream endpoint returned an error, including rate-limit responses. Record the normalized state and the raw status.
• Client or gateway timeout: No complete response arrived within the configured limit. Record which component timed out and the limit applied.
• Mid-stream disconnect: The stream ended abnormally after some output was sent. Record how much was delivered and to whom.
• Content truncation: Output ended at a token or length limit or a provider content stop. Record the stop reason.
• Policy-blocked or redacted: Runtime enforcement withheld or altered the request or response. Record the decision and policy.
• Retry or fallback: A repeated or rerouted attempt. Record its link to the prior attempt.

Minimum audit record for each state

Use one record format across services so events can be consumed and correlated by many systems. Keep the core record small and structured, and move extended data such as request and response bodies, headers, and stack traces into separate tables or files referenced by identifier. That separation also simplifies access control and retention for sensitive content.

The core fields below apply to every state. State-specific fields extend them, for example the timeout limit and elapsed time for timeouts, or bytes and tokens delivered for interrupted streams.

• Identity and time: Interaction ID, attempt ID, tool-call ID where relevant, timestamps for start and end, and the authenticated user or calling system.
• Target: Model or tool addressed, provider endpoint, and deployment region.
• Outcome: Normalized state plus the raw provider or gateway signal.
• Decision: Allow, redact, block, rate limit, or escalate, and the identifier and version of the policy that triggered it.
• Delivery: What the user or downstream system actually received: nothing, partial, redacted, or complete, with a content hash.
• Payload reference: Pointer to separately stored extended data, with its retention class.

Where in the request path capture should occur

Capture at more than one point, but treat one point as authoritative. Application code sees its own exceptions. It does not reliably see what a gateway cut off, what a stream delivered before dropping, or what a policy decided. Records written only from application error handlers tend to differ by service and omit the decision.

The authoritative point is the component that enforces policy in the execution path, because it observes the request, the decision, the upstream result, and the bytes forwarded to the caller in one place. Application logs remain useful for diagnosis, but they are a different artifact from evidence produced by enforcement. Keep audit and transaction logs separate from security event logs where their purposes differ, and forward records from distributed components to a central, secure logging service.

Decide explicitly what happens when the audit write itself fails. Guidance for agent systems is to fail closed when policy lookup or audit logging fails. The tradeoff is availability: a request that cannot be recorded is refused. For consequential use cases that is usually the correct trade, but apply it by risk tier and document the choice.

Sensitive data, retention, and integrity

Partial outputs are awkward because they end at arbitrary points. A cut-off stream can contain half of an identifier or a fragment that a detector would have caught in full. Apply redaction before storage, run it on the accumulated text rather than on each chunk alone, and store the delivered content, not the intended content. Record that redaction occurred and which policy caused it. Model inputs and outputs are untrusted text, so encode and validate dangerous characters before writing them, to prevent log injection.

Retention and data residency should be set per record class, not globally. Core metadata and extended payloads can carry different retention periods and may need to stay in specific regions. The relevant periods come from your own legal and regulatory obligations, so define them in policy and enforce them in storage.

For integrity, restrict permissions on log storage and keep a change audit of the logs. Hashing delivered content and chaining or signing records are common ways to make tampering detectable, and write-once storage is a common way to prevent it. Choose based on the assurance your auditors expect.

Correlating retries and duplicates, and where runtime enforcement fits

Give each logical interaction a stable interaction ID and each tool call its own ID. Every attempt then gets a unique attempt ID, an attempt number, a reference to the attempt it follows, and a reason code such as timeout, provider error, or fallback to another model. An auditor can then see that three attempts produced one delivered response, or that a retry duplicated a tool call with side effects. Record the idempotency key if one is used.

This correlation applies to individual interactions and tool calls. It is not end-to-end tracing of an agentic workflow, and the audit record should not claim to be one.

Trussed enforces policy in the execution path across models, agents, tools, and MCP servers, with decisions to allow, redact, block, rate limit, or escalate for human approval, scoped to individual interactions and tool calls. Because the decision is enforced at runtime, not reviewed after the fact, non-success outcomes produce audit-ready evidence as a by-product of enforcement. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency. Its evidence is mapped to HIPAA, NAIC, NYDFS, the EU AI Act, and NIST, and it has run in continuous production in a regulated enterprise for over 18 months.