What Is a Token in LLM Pricing, and How Does It Relate to Governance?

What a token is and how tokenization works

In LLM usage, a token is a unit of data, such as text, characters or phrases, that a model uses to read, understand and process language. Tokenization is the step that converts raw inputs into numerical sequences the model can work with. This applies beyond text: images, audio and video are also converted into token sequences. Every input and every output becomes a sequence of tokens.

The word is overloaded. In security vocabulary, including the NIST glossary, a token is a portable, user-controlled physical device such as a smart card or memory stick that stores cryptographic information. That meaning has no connection to an LLM token, and architecture documents that cover both authentication and model usage should keep the terms separate.

Because tokenization is part of how a specific model processes input, the count for a given piece of text depends on the tokenizer in use. A count produced for one model should not be assumed to carry over to another. Any limit, budget or comparison expressed in tokens needs the model identifier attached to it.

How tokens are metered

Organizations using AI services typically pay based on usage measured in tokens, rather than per user or per licence. Charges generally reflect the amount and type of input and output, as well as the processing performed. Metering distinguishes three categories. Input tokens are those sent into the model. Output tokens are those generated in the response. Cached tokens are a subset of input tokens that let the model reuse input from previously processed prompts.

Consumption is hard to predict because it varies with prompt length, response length, the context provided to the model and the specific capabilities used. A short user question can sit on top of a long system prompt, retrieved documents and conversation history, all of which count as input. In agent loops, each model call carries its own input and output, so a multi-step task is the sum of many interactions. Tool calls add further requests and results that flow back into the context.

Providers may report additional categories, such as reasoning or tool-related tokens, and may price each category differently. Treat every category as a separate field to capture, and record it with the model identifier so that per-interaction records stay interpretable.

Where tokens intersect with governance

Token-based controls are useful, but they operate on volume. Context window limits cap how much can be sent in one request. Per-request maximums cap the size of a single interaction. Tokens-per-minute rate limits cap throughput over time. None of these inspect what the tokens contain or which action they trigger.

That gap matters in practice. A request well inside every limit can still carry sensitive data in a prompt, and a short tool call can still be unauthorized. Context exposure, meaning what was placed in front of the model, is a governance question that a count cannot answer. The table separates three control levels so each can be assigned the right job.

• Token-level controls: Act on count: context limits, per-interaction maximums, rate limits. They bound volume and throughput but do not evaluate content, intent or permitted action.
• Content-level policy: Acts on what is sent to and returned from a model, for example allowing, redacting or blocking specific content.
• Tool-level policy: Acts on which tool or MCP server an agent calls and with what arguments, independent of how many tokens the call consumes.
• Human escalation: Routes a specific interaction or tool call to a person for approval when the policy requires a decision that automation should not make.

Runtime policy enforcement and the evidence it produces

Runtime enforcement acts on the token stream as requests and responses pass through the execution path. For each individual interaction or tool call, a policy can allow it, redact part of it, block it, rate limit it, or escalate it for human approval. The decision can rest on token count, on content, or on tool identity, and a mature policy usually combines them. Enforced at runtime, a decision takes effect before the interaction completes, rather than being reviewed after the fact.

The scope should be stated precisely. Enforcement at the level of individual interactions and tool calls is different from governing a business transaction across multiple tasks and agents, and different from tracing an entire agentic workflow end to end. Evaluate any tool against the first scope without assuming the others.

Enforcement also generates the evidence. A useful interaction record includes what was sent to the model, what was returned, the policy decision applied, the model identifier and the token counts by category. Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with 100+ models under one policy plane, and produces continuous audit-ready evidence mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST.

On the regulatory side, the NIST AI RMF can be used to prioritize risk but does not prescribe risk tolerance, so each organization decides what its token-level and content-level controls should enforce. As of October 2026, the EU AI Act requires member states to provide for penalties and other enforcement measures, including for supplying incorrect, incomplete or misleading information to authorities. An accurate, retained record of what an AI system was permitted to do and did is a sound basis for answering such questions.