Exporting auditor-ready evidence from LiteLLM logs

Start from what the auditor must reconstruct

Auditors and compliance reviewers usually ask the same questions about AI activity: which requests occurred, who made them, which models and tools were involved, what policy applied, and what happened as a result. LiteLLM proxy logs are produced for operations and debugging, so starting from the log format tends to produce an export that is large and still incomplete. Start from the questions instead.

This follows the idea of forensic readiness: define the digital evidence needed to reconstruct past events of interest, then collect it at low cost. It also matches NIST guidance that audit logs and records should be created and retained to the extent needed to support monitoring, analysis, investigation and reporting. NIST log management guidance lists auditors as users of log data, which means the export has to be interpretable by someone who did not build the system.

Write the requirements down as a short evidence specification before touching the export. The specification becomes the contract between platform engineering and the reviewer, and it makes gaps visible early.

Map each evidence field to a LiteLLM source

For every requirement in the specification, identify which request log, spend record or logging callback payload in your deployment supplies it. If nothing does, record the field as missing rather than inferring it. An auditor can accept a documented limitation more easily than an undocumented assumption.

Gateway records describe requests that passed through the proxy. Whether they also show the policy that applied or the outcome depends on what your configuration captures, so verify this per field. Decision criteria are a common gap, since audit trails for AI systems are expected to cover content flows and decision-making criteria, and operational logs are not written with that in mind.

• Identity: The user, key or team behind each request, and how that identity maps to a person or service owner.
• Model: The model requested and the model used, with version information where your records hold it.
• Time: A consistent timestamp for each request, with the time source and zone documented.
• Tools: Which tools or functions were invoked, if your records capture them.
• Outcome: Whether the request completed, failed or was refused, and by what mechanism.
• Policy applied: The rule or control in effect for the request. Mark as missing if logs only show that the request occurred.

Handle prompt and response content deliberately

Prompts and responses can contain personal, privileged or otherwise sensitive data. Guidance for LLM systems advises logging only the minimum interaction metadata needed for security monitoring, and minimizing, redacting or anonymizing any prompt or output content before it is stored. The point is timing: redacting at export time still leaves the sensitive content in the original store.

A practical split is to treat metadata as the primary evidence set and content as an exception. Metadata answers who, what model, when and what outcome. Content is added only where a specific review requires it, with a defined redaction step, a restricted audience and a stated reason. Document which fields are redacted and how, so the reviewer understands why a record looks the way it does and does not read it as incomplete.

Where log-derived exports fall short

A well-built export can reliably show that a request occurred, who or what key made it, which model was addressed and when. That is useful evidence of activity, and for many review questions it is enough.

The limits appear when the question changes from what happened to what the system was permitted to do. Evidence reconstructed from logs after the fact is an interpretation. The reviewer has to trust that the policy you describe was the policy in force at that moment, and that nothing outside the logged path changed the outcome. If the records do not capture which rule applied or whether content was allowed, redacted or blocked, the export cannot demonstrate it, however carefully it is packaged.

There are also operational limits. Logs written for debugging may omit decision criteria, and tamper evidence is something you add around the logs, not something they provide on their own. Retention, access control and chain of custody for copies given to auditors remain manual processes that need an owner.

How runtime enforcement produces evidence

The gap above is the difference between recording requests and enforcing policy. Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with actions that include allow, redact, block, rate limit and escalate for human approval. Enforcement applies to individual interactions and tool calls. Because the decision is made at runtime, not reviewed after the fact, the record of that decision is generated by the enforcement itself.

That changes what an auditor receives. Instead of inferring which policy applied from request logs, the reviewer sees audit-ready evidence of what the AI was allowed to do and what it did, mapped to frameworks including HIPAA, NAIC, NYDFS, the EU AI Act and NIST. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency, which bears on the access, retention and sensitive-content decisions described above.

This is a complement to a careful log export, not a replacement for the steps in this guide. Trussed does not provide end-to-end tracing of agentic workflows, transaction-level governance or model lineage, so those questions need other evidence.