Investigating an AI Spend Incident and the Records Required to Reconstruct What Happened

Why a spend spike is an evidence problem

A runaway agent loop, a misused credential, an unapproved model or an unusually large prompt volume all look the same on an invoice: consumption went up. The invoice cannot tell you which of those happened. The questions an investigator, auditor or regulator asks are different: who or what initiated the activity, which policies applied, and what each interaction or tool call did.

Treating the event as a billing problem leads to a cost explanation. Treating it as an evidence problem leads to a reconstruction. The second is what a security team needs, and it depends on records that are often scattered across provider exports, API key inventories, application logs and identity systems, with inconsistent retention and no shared identifier linking them.

OWASP describes AI observability data as the forensic record that incident response depends on, and frames reconstruction as three questions: what the model received, what it produced and what actions it took. Interaction-level records need to answer all three.

Provider records versus records you must generate

Provider usage and billing exports describe consumption from the provider's side. They are useful for detection and scoping, but they reflect what the provider can see: the key used, the model, the volume and the time. They generally do not show your internal caller, which policy applied, or what an agent did with the response.

The records that close the attribution gap are generated by the enterprise at the point where requests are made. That means interaction-level and tool-call-level records written in your own environment, retained under your own settings, and correlated to provider data through shared identifiers. The correlation mechanism is a design decision you need to make before an incident, for example a request identifier carried from the caller through to the provider call and into your own records.

Identity deserves particular care for agents. Stakeholder comments to NIST's NCCoE on agent identity note that user and agent identity, authorizations and context may need to travel with the agent across boundaries. In practice, record both the agent identity and the user or service on whose behalf it acted, at the point where the action occurs.

Judging whether evidence is durable and audit-ready

Having the fields is not enough. Evidence has to survive until someone asks for it and has to be credible when they do. Three properties matter: retention, location and integrity.

Retention should cover the period over which an investigation, audit or inquiry could reasonably occur, not the incident window alone. The EU AI Act points the same way for high-risk AI systems, which must technically allow automatic recording of events over the lifetime of the system. Location matters because data residency constraints may limit where records can be stored and who can access them. Integrity matters because records that can be altered after the fact carry less weight. Ask who can modify or delete each record class and whether changes are detectable.

Where the EU AI Act applies, reporting timelines add pressure. As of October 2026, providers of high-risk AI systems must report a serious incident immediately after a causal link, or reasonable likelihood of one, is established, and no later than 15 days after the provider or deployer becomes aware. Deployers that identify a serious incident must inform the provider first, then the importer or distributor and the relevant market surveillance authorities. Whether a given spend or usage anomaly is a serious incident is a determination to make in advance, with defined roles for notification, so that your records can support the report if one is required.

Testing your records against a real or simulated spike

The most reliable way to find gaps is to run a reconstruction. Take a past spike or simulate one, such as a looping agent or a key used outside its intended application. Then try to answer, using only retained records, who or what initiated it, which policy applied, and what each tool call did.

Apply the received, produced and acted test to each interaction. If you can show what was sent but not what came back, or you can see the tool call but not the identity behind it, record the gap and its owner. Also check the joins: can a provider billing line be matched to an internal request without manual guesswork? Time the exercise. A reconstruction that takes weeks will not meet an urgent regulatory or contractual deadline.

Finally, confirm that the incident record itself is complete. Containment actions, approvals and decisions made during the response belong in the same evidence set as the technical records.

Where execution-path enforcement fits

Records written after the fact depend on every application remembering to produce them. Enforcing policy in the execution path changes that: each decision is made, and recorded, when the interaction or tool call occurs.

Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants. Each interaction or tool call is allowed, redacted, blocked, rate limited or escalated for human approval, enforced at runtime rather than reviewed after the fact. That enforcement produces audit-ready evidence of the policy applied and the decision taken, at the level of individual interactions and tool calls. The evidence is mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency, and generates 30+ GB of audit evidence per day in production use.

This addresses the policy-decision, rate-limit and escalation records in the checklist above. It does not remove the need for identity records, provider data and shared identifiers, which the investigation still depends on.