Adding Tamper-Evident Audit Logs for Azure OpenAI Behind API Management

What tamper-evident means, and where native logging stops

Tamper-evident does not mean tamper-proof. It means any modification, deletion or truncation of the record becomes detectable. NIST SP 800-209 describes log tampering as an attacker deleting or modifying log data to prevent an effective audit trail, conceal an attack, or mislead investigators with false information. That is the threat model for this design.

Most teams start with gateway diagnostics and Azure Monitor. Those are useful operational records, but they usually live in the same administrative plane as the gateway and the people who operate it. A privileged administrator can change a retention setting or a diagnostic configuration. Capture can also be partial: prompt and completion bodies, streamed responses and policy decisions are the fields auditors ask about and the ones most often missing or truncated. An auditor asking whether the record is complete and unaltered needs more than the platform's own assertion.

NIST SP 800-171r3 sets the expectation: protect audit information and audit logging tools from unauthorized access, modification and deletion, and limit management of audit logging functionality to a subset of privileged users or roles. The rest of this guide turns that into a build.

Step 1: Decide what to capture and who decided it

Specify event types and record content as an explicit, documented decision. NIST SP 800-53r5 and SP 800-171r3 leave these as organization-defined, so record the rationale along with the list.

The NIST SP 800-228 minimum API telemetry fields are a floor: timestamp, client ID, request and response status, and endpoint identifier. An LLM audit schema needs more. SP 800-53r5 also notes that records can include access control or flow control rules invoked, which maps directly to gateway policy decisions. A workable schema extends the floor with these fields:

• Record sequence number, previous record hash and record hash
• Caller identity, subscription or client ID, and the deployment or model endpoint called
• Request content or a reference to it, with the masking applied
• Response content or a reference to it, with completion status (complete, aborted, truncated)
• Policy decision: which rules ran and whether the outcome was allow, modify or reject
• Gateway timestamp and the identifier of the component that wrote the record

Step 2: Chain, sign and store the records

Have the gateway policy emit each audit event to a dedicated writer rather than relying on the diagnostic pipeline. The writer serializes each record in a canonical form, so the same content always produces the same bytes, and computes the record hash over that serialization plus the previous record's hash. Changing, removing or reordering any record then breaks every hash after it. Use a standard hash such as SHA-256.

A chain alone can be rebuilt by someone with write access. To prevent that, sign a checkpoint (the latest chain hash, sequence number and time range) at regular intervals. Hold the signing key in a key store where the logging operators cannot export it, and where the people being audited cannot use it. Publish checkpoints to a second location with different owners. NIST SP 800-92 makes the same point for digests: store them securely and apart from what they protect, or changes cannot be reliably detected.

Write the records and checkpoints to a central store that is separately controlled from the gateway subscription, as SP 800-209 recommends for central log servers. Use write-once storage with a retention lock where your platform offers it, and size capacity against your defined retention period, as SP 800-53r5 requires. Evaluate any ledger-style service on the same criteria: who can administer it, what it prevents, and what it leaves to you.

Separation of duties ties these together. One role operates the gateway, another administers the audit store, and a third controls signing keys. No single role should be able to alter a record and also regenerate the evidence that covers it.

Step 3: Handle streaming and sensitive content

Streaming responses arrive as many fragments, so a per-request log line cannot capture them unless the gateway assembles them. Decide the rule up front: write one audit record per interaction when the stream ends, containing the assembled response, and mark the record explicitly if the client disconnected, the stream errored or the content was cut short. A missing record for a stream that never finished is itself a gap, so write a provisional record when the request starts and a closing record when it completes. The sequence numbers will then expose any stream that vanished.

Prompts and completions often contain personal or regulated data. NIST SP 800-228 recommends data masking in responses and logs, but auditors still need to see what happened. Two patterns balance these needs. First, store the masked content in the main record along with a digest of the original, so you can later prove a retained original matches. Second, keep the original in a separate, encrypted store under tighter access and shorter retention, referenced by ID. In both cases, hash the content as stored so masking choices are themselves part of the evidence, and record which masking rules ran.

Where runtime policy enforcement adds to this pattern

The pattern above makes the record of what happened trustworthy. It does not decide what is allowed to happen. A gateway policy can reject a request, but a record assembled afterward is still a review of past events, and the policy decision is only as complete as what you chose to emit.

Trussed enforces policy in the execution path: allow, redact, block, rate limit or escalate for human approval on individual interactions and tool calls. The audit-ready evidence is generated by that enforcement, so it is enforced at runtime, not reviewed after the fact. Evidence is mapped to frameworks including HIPAA, NAIC, NYDFS, the EU AI Act and NIST, and Trussed can be deployed self-managed in the customer cloud with configurable retention and data residency. For a regulated enterprise, that means the decision and its evidence come from the same point, rather than being reconstructed from gateway diagnostics.