Why AI Agents Make LLM Costs Unpredictable

Why request counts stop working as a forecast

A conventional LLM application has a roughly fixed shape. A user action produces one request, or a small and predictable chain of them, and token use per request clusters around a typical prompt and response size. Request counts, seat counts or traffic forecasts are reasonable proxies for consumption because the work per request is bounded by the application's code.

An agent moves that decision into the model. Agents choose which tools to call, make repeated LLM calls, and use prior outputs to guide subsequent actions. The number of model calls for a task is therefore a runtime outcome rather than a design constant. Two tasks that enter the system as one request each can consume very different amounts of inference.

This variability is consistent with how NIST describes AI more broadly: outputs are typically non-deterministic, so the same input conditions can produce a range of behaviors. For agents, that range includes how long the task runs. The link from non-determinism to token consumption is an engineering inference rather than a measured result, but it follows directly from the structure of the loop.

Five mechanisms that drive variability

Autonomous loop depth. A common agent design is a ReAct-style loop that repeats until the goal is achieved, the agent chooses to submit a solution, or the budget allotted for the task is exhausted. Each pass is at least one model call. When the termination condition is judged by the model, the depth of the loop is not known in advance, and a task that fails to converge keeps consuming until something external stops it.

Retries and error recovery. When a tool call fails, returns an unexpected format or produces an unsatisfying result, the agent or its framework typically tries again, often with a revised plan. Every attempt repeats the cost of the reasoning step that preceded it. A degraded downstream service can raise consumption across many tasks at once without any change in request volume.

Context accumulation. Agents rely on short-term context windows and long-term memory stores such as summaries, retrieval entries, cached state and stored outputs. As a task proceeds, tool results and prior reasoning are carried forward, so later steps process more input than earlier ones. Consumption per step rises across the task, which means total cost grows faster than the step count alone suggests.

Tool and MCP call chains. Agents select tools from natural-language tasks and can dynamically load third-party tools, MCP endpoints and other agents. Each tool result re-enters the model as input and can prompt further calls. Which tools are used, in what order and how many times is decided during execution.

Multi-agent delegation. When an agent hands work to sub-agents, each sub-agent runs its own loop with its own context. Fan-out multiplies the other four mechanisms. NIST material also notes that agents each acting within their own specification can still converge on outcomes that no single component would produce alone, and the same compounding applies to consumption.

What enforcement in the execution path can constrain

NIST's guidance on agent identity frames control at the level of the individual tool invocation, API call or resource access, verified continuously at runtime against current state, context and risk. Policy enforcement in the execution path applies the same idea to model interactions: a decision is made on each call before it proceeds, not reviewed after the fact.

At that level, the available decisions are concrete. An interaction or tool call can be allowed, redacted, blocked, rate limited, or escalated for human approval. Scoping policy per agent, tool or MCP endpoint follows least privilege, which NIST recommends for agents: grant only the permissions the role needs and avoid broad privileges or open-ended tools.

Mapped to the mechanisms above, this gives practical levers. A rate limit on model interactions or on a specific tool bounds how fast a runaway loop or retry storm can proceed. A block on a tool outside an agent's role removes an entire branch of a call chain. Escalation for human approval places a person in front of a high-impact or unusual call. Deterministic decisioning at these points also gives more consistent behavior at the boundary, even when the agent behind it is not deterministic. That describes the control surface, not a guaranteed spend ceiling.

Where per-interaction enforcement stops

A control that evaluates one interaction or tool call at a time sees that call. It does not by itself reason about the workflow the call belongs to. Several of the mechanisms above only show up across steps: context that grows cumulatively, authorizations that aggregate as a task chain executes, and multi-agent behavior that emerges from the interaction of components. Each individual call can be within policy while the sequence is not.

This matters for how teams scope expectations. Per-call limits constrain the rate and permitted shape of behavior at each point, but they are not end-to-end tracing of a workflow and they are not transaction-level governance across tasks and agents. Workflow-level questions, such as why a particular task consumed what it did across several agents, need other controls: task budgets and iteration limits in the agent framework, telemetry that is never omitted, and post-deployment monitoring. Monitoring has its own resource cost, and NIST notes it may be higher for agents.

The sensible architecture layers these. Framework-level caps shape the workflow, and enforcement in the execution path provides an independent boundary on each call that does not depend on the agent behaving as designed.

Evidence of what the agent was allowed to do

For a regulated enterprise, limits matter in two ways: they constrain behavior, and they must be demonstrable. A control that only observes cannot show that a call was prevented. A control in the path can record each decision as it is made, including which calls were allowed, rate limited, blocked or escalated, and who approved an escalation.

The direction of travel in public guidance supports this. NIST suggests agents may need new policies governing permissions and authorizations, with continuous runtime verification of tool calls. A GSA acquisition regulation document published in the Federal Register in June 2026 contains a provision calling for human oversight, intervention and traceability, and for intermediary steps of agentic processes to be accessible through an audit trail. Teams should confirm status and applicability from the full text as of October 2026.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers, and the enforcement itself generates audit-ready evidence mapped to frameworks including NIST and the EU AI Act. Its scope is individual interactions and tool calls.