Why LLM Costs Are Hard to Predict in Enterprise Deployments, and Which Drivers Policy Can Bound

Why spend does not track request volume

A single LLM request is priced on the tokens it consumes: input tokens, which include the prompt and any context, and output tokens, which the model generates. Neither count is fixed. Two requests to the same endpoint can differ by orders of magnitude in size, so a forecast built on request counts or seat counts has no stable unit cost to multiply.

Model choice adds a second layer. NIST CAISI notes that per-token costs can be poorly correlated with end-to-end expense when different models use different numbers of tokens to complete the same task. A cheaper model that needs more tokens, or more attempts, to finish a task is not necessarily cheaper in practice. Providers also structure pricing differently, so the same workload can map to different bills depending on where it is routed.

Several other mechanics widen the spread. Context grows when conversation history, retrieved documents or tool results are carried into each subsequent call. Retries repeat a request, and its tokens, after a failure or an unacceptable response. Routing decisions can move traffic between models with different token behavior and prices. Finally, usage appears outside planned workloads, such as a team adopting a coding assistant or connecting a new tool without a capacity estimate.

How agents and coding assistants change the shape of the problem

In a single-turn application, one user action produces roughly one model call. Agents do not behave that way. NIST describes agents as systems that gain agency through extensions, tools or plugins, dynamically choose among them, and make repeated LLM calls that use prior outputs to guide the next action. The number of calls and the token volume are therefore decided at runtime, not fixed when the request arrives. Request counts are a weak proxy for cost in these systems. That conclusion is an inference from how agents work, not a measured forecast error.

Tool use compounds this. When a model is embedded in scaffolding that lets it manipulate tools, one user task can span many model and tool interactions, and each tool result typically returns to the model as more context. MCP servers are one route by which such tools are exposed.

Coding assistants span a range of autonomy, from autocomplete to interactive agents to autonomous agents. A study of these tool generations notes that the cost of using AI has a direct inference component and a time component covering prompting, configuration and review. For forecasting, it is sensible to treat agentic and coding-assistant workloads as separate classes from single-turn requests, each with its own usage expectations.

What policy in the execution path can bound

NIST's agent identity design tenets separate the logic that decides what an agent attempts from the systems that decide what it is allowed to do. The same separation applies to usage. If limits live inside the prompt or the agent's reasoning loop, the model can be steered around them. If they are enforced at runtime in the execution path, each interaction and tool call is evaluated against policy before it proceeds.

At that level, several controls bound exposure. A rate limit caps how often an identity, agent or tool can call within a period, which limits retry storms and runaway loops. Blocking stops a request or tool call that policy does not permit, such as an unapproved tool or an out-of-scope action. Escalation for human approval pauses specified calls until a person decides. Redaction removes sensitive content from an interaction before it moves on, which is a data control and should not be counted as a usage control.

Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants, with allow, redact, block, rate limit and escalate for human approval, scoped to individual interactions and tool calls.

The tradeoff is operational. A limit that is too tight fails legitimate tasks, and one that is too loose bounds little. NIST also notes that introducing expense limits on models can change conclusions about cost efficiency, so limits should be tested against the workloads they affect.

What runtime enforcement cannot predict, and what evidence it adds

Enforcement sets a ceiling. It does not tell you what usage will be. A rate limit can cap calls per period, but it cannot estimate how many tokens a task will need, how prices will change, or how many users will adopt a tool. Forecasting remains a separate modeling exercise, and Trussed does not forecast spend. Trussed also does not provide end-to-end tracing of agentic workflows, so the full chain of a multi-step task has to be reconstructed by other means.

What enforcement does produce is a record of decisions. Audit-ready evidence generated by enforcement shows which policy applied to an interaction or tool call and whether it was allowed, redacted, blocked, rate limited or escalated. Provider billing data answers a different question: what was consumed, aggregated by the provider's own dimensions. Reconciliation uses both. Billing shows the totals, and enforcement evidence helps explain which calls were permitted, which were stopped and which required approval.

For a regulated enterprise, that record also serves oversight. Draft federal procurement text published for comment in June 2026 refers to human oversight, intervention and traceability. As of October 2026 that text is a proposal, and it is framed around Government Data protection, not usage reconciliation. Trussed evidence is mapped to HIPAA, NAIC, NYDFS, EU AI Act and NIST, and is generated continuously at production volume.