Azure API Management LLM diagnostic logs: what they capture and what they miss for auditors

Operational telemetry is not the same as audit evidence

When LLM traffic is routed through Azure API Management, auditors are often handed diagnostic logs as proof of AI activity. These logs are built to help operators run the service: they show that calls happened, roughly how large they were, how many tokens were consumed and what the gateway did at the transport level. That is useful for capacity, performance and cost questions, and token and usage records also matter for resource-management risks such as the Unbounded Consumption category in the 2025 OWASP LLM Top 10.

An audit asks different questions. Who called which model, what was sent and returned, which policy applied, what was allowed, redacted or blocked, and can the record be trusted as complete and unaltered? A log designed for operators answers some of these only partially. The useful first step is to stop asking whether logs exist and start asking which audit question each field can answer.

What the logs typically capture

As a baseline, gateway diagnostic logs for LLM calls typically carry request metadata, token usage and gateway-level events. In practice that means a timestamp, the API or operation called, a response status, latency and token counts, and some form of caller or subscription context. The exact fields depend on how diagnostics are configured, so the schema in the live environment is the reference, not a generic description.

A helpful yardstick comes from OWASP, which describes AI observability as covering input logs, output logs and tool call traces. Metadata and token counts address the volume and timing of activity. They do not, on their own, cover inputs, outputs or tool calls. Comparing the actual log schema against those three categories quickly shows how much of the expected scope is present.

Configuration choices that change what is recorded

Four configuration areas determine whether a log can serve as evidence. Each should be verified from the configuration in effect, not inferred from a sample export.

• Payload logging: Decides whether prompt and response content is recorded at all. If it is off, the log cannot show what was sent to or returned from the model.
• Sampling: If only a share of calls is recorded, the log cannot support completeness claims about all interactions.
• Size limits: Large prompts or responses may be truncated, so a recorded payload may not be the full payload.
• Destination: Where logs are sent determines who can read, alter or delete them, and what other systems hold a copy.
• Retention: Records must still exist when the audit period is examined. Retention that is shorter than the audit window leaves silent gaps.
• Log level and tagging: OWASP notes that poorly set levels and inappropriately tagged data can bury critical data or leak sensitive data into production logs.

The tension between content and exposure

Recording prompt and response content closes one gap and opens another. If prompts contain personal or confidential data, a content-rich log becomes a sensitive store of its own, with access, residency and retention obligations. OWASP's secure logging guidance makes the same point: data that is inappropriately tagged can leak into production logs.

Auditors should therefore expect a documented decision rather than a default. Either content is recorded and protected accordingly, or it is deliberately excluded and other evidence shows what was exchanged. A log that omits content with no stated rationale, or captures it with no access controls, is a finding in either direction.

Where evidence gaps typically arise

The first gap is content. Without payload records, an auditor can see that a call occurred and how many tokens it used, but not what the model was asked or what it returned.

The second is policy. Gateway events show that a request passed or failed at the transport level. They do not necessarily show which policy applied to a specific interaction, or whether the outcome was allow, redact or block. The third is tool calls. OWASP lists tool call traces as part of AI observability, and a metadata-only log generally will not contain them.

The fourth gap is the most important for assurance: permitted versus actual. A log can describe what happened, but showing what an AI system was allowed to do requires a record of the rules in force at the time and the decision made against them. Without that, an auditor can describe activity but cannot test it against intent.

Finally there is integrity. OWASP names verbose logs that lose critical data in noise, and logs that are overwritten, as common pitfalls, and advises designing logs for the future forensic reader. Auditors should ask how completeness, retention and protection from alteration are demonstrated, not assumed. Where a regulation or framework such as the EU AI Act or the NIST generative AI profile applies, its record-keeping expectations should be mapped to specific fields before the logs are accepted as evidence. As of November 2026, the Commission's Digital Omnibus on AI, which would delay high-risk application dates, is a proposal, so timelines should be confirmed before they are relied on.

When runtime policy enforcement evidence is needed

If the audit question is only about usage, volume or availability, gateway diagnostics may be sufficient once their configuration is verified. If the question is what an AI system was permitted to do and what it did, a separate record is needed: one produced at the moment a policy decision is made, for each interaction and each tool call.

Trussed addresses this case. It enforces policy in the execution path across models, agents, tools and MCP servers, applying allow, redact, block, rate limit or escalation for human approval to individual interactions and tool calls. The enforcement itself generates audit-ready evidence, mapped to frameworks including the EU AI Act and NIST, so the record of each decision is a product of enforcement at runtime, not reviewed after the fact. Deployment is self-managed in the customer cloud, with configurable retention and data residency.

This complements gateway telemetry rather than replacing it. Diagnostic logs describe traffic. Enforcement evidence describes decisions. An audit file that holds both can answer who called, what was allowed and what happened.