How to Budget an AI Use Case Without Usage Data

Decompose the cost drivers before estimating

A single total hides which assumption is wrong. Start with a list of drivers and give each one an owner and a stated basis. Under usage-based model pricing, spend scales with the number of requests, the tokens sent and returned, and the size of the context attached to each request. Context size depends on prompt design, retrieved passages, and conversation history, so it varies by user and by task.

Agentic designs add a multiplier. One user request can produce several model calls and tool calls, and each tool call can invoke downstream services such as external APIs, databases, or compute jobs. Those costs sit outside the model's token bill, so an inference-only estimate omits them. Retries multiply both categories.

The drivers most exposed to uncertainty are the ones that depend on behavior you have not yet observed: fan-out per request, retry share, context size, and adoption. List prices are known. Behavior is not. Spend your modeling effort accordingly.

• Model inference: Requests multiplied by input and output tokens under the provider's pricing structure.
• Context variability: Prompt, retrieval, and history size per request, modeled as a range.
• Agent and tool-call fan-out: Model calls and tool calls per request, including downstream service consumption.
• Retries: Share of calls repeated after failure, timeout, or rejected output.
• Retrieval and storage: Index, vector store, and data storage that scale with corpus and retention.
• Human review effort: Reviewer time per escalated item multiplied by expected escalation volume.

Express demand as scenario ranges and bound the high case

Without observed volumes, a point estimate gives false precision. Build three scenarios instead. For each, set a value for every driver: requests per day, average tokens per request, calls per request, retry share, and the share of items sent to human review. Keep the low, expected, and high scenarios internally consistent, so a high-adoption case also carries a higher review load.

Where the organization has a comparable internal workload, use it as a reference class and record it as the basis for the expected case. Where it does not, say so and widen the range. The point is to make each assumption visible so finance and architecture can challenge it.

The high scenario needs an upper bound that is enforced, not assumed. Agents that can loop or chain tool calls without a per-session cap on API calls, tokens consumed, or wall-clock time have no natural ceiling on consumption. Set those per-session limits, place them in the execution path rather than relying on application code, and use them to cap the high case. Treat these limits as risk and scope boundaries. They define what the use case is permitted to do in one session. They are not a savings mechanism, and the budget should not count them as one.