AI Unit Economics: Cost per Query and Cost per Resolution
A practical framework for fully loaded AI unit economics: cost components, resolution quality, attribution, and runtime spend controls for enterprise agents.
AI cost per resolution is the fully loaded cost of delivering a business outcome that meets explicit success criteria, including model inference, retrieval, tool calls, platform overhead, security controls, and human escalation. Cost per query is the attributable spend for a single request through the agent stack. Enterprises should instrument multi-step traces, allocate shared costs by clear keys, validate resolution quality beyond model success, and enforce runtime budgets and least-privilege tool access to control volatility.
From token bills to outcome economics
Token list prices alone do not describe the economics of agentic systems. Use these four constructs when you move from invoice line items to outcome metrics.
Cost per query
Fully loaded variable and allocated fixed cost for one request across ingress, models, retrieval, tools, and egress.
Cost per resolution
Cohort cost divided by outcomes that pass task-success, quality, and policy checks, including human steps.
Primary drivers
Token volume, multi-step loops, tool retries, large context, and unconstrained high-cost actions.
Control layer
Telemetry attribution, budgets, max steps and tokens, tool allow lists, and graceful degrade paths.
Why token price is not unit economics
Enterprise teams often treat list prices for input and output tokens as the cost of AI. That understates agentic systems. Cloud model serving is commonly billed on tokens, and sometimes on provisioned throughput or dedicated capacity, so context length and completion size matter. Retrieval-augmented and multi-step agents add embedding generation, vector or knowledge-base queries, repeated model calls, and external tool or API invocations. On top of that sit orchestration runtimes, observability storage, guardrails, identity and secrets, network egress, and human handling when automation does not finish the job.
FinOps practice for AI and ML pushes allocation of spend to products, teams, and features, and favors unit metrics that connect invoices to business value: cost per transaction, workflow, or outcome. For agents, the meaningful outcome metric is cost per successful resolution, not cost per completion. Without a written cost model and consistent success definitions, dashboards drift from billed spend and optimization work targets the wrong layer of the stack.
Defining cost per query and cost per successful resolution
Cost per AI query is the fully loaded variable and allocated fixed cost attributable to one user or system request as it traverses the agent stack: ingress, orchestration, one or more models, retrieval, tools, and egress. It is useful for comparing model routes, prompt designs, and retrieval strategies on a per-request basis.
Cost per successful resolution is different. It divides the fully loaded costs for a defined outcome cohort by the count of resolutions that meet explicit success criteria. A resolution may span multiple queries, tool sequences, sessions, and human steps. Containment or deflection metrics from contact center and IT service operations are a useful analogy: partial AI assist still incurs human labor, vendor seats, and follow-up work, so those costs belong in the fully loaded figure when automation is incomplete.
There is no single industry standard formula. Each enterprise must publish inclusion rules for direct versus allocated costs, allocation keys such as token share, vCPU-hours, or seat count, and a refresh cadence tied to cloud invoices. Unit models also need ongoing calibration because vendor pricing units and commitment constructs change frequently.
Working distinction
Use cost per query to compare routes and prompts on a single request. Use cost per successful resolution to judge whether automation is creating business value after quality, policy, and escalation costs are included.
Cost components to include
Direct components typically include LLM input and output tokens, embedding tokens, managed retrieval or knowledge-base queries, tool and API charges, speech or multimodal units when used, and premium model endpoints. These map cleanly to rate cards and should be metered per request and per step.
Indirect and shared components typically include GPU or provisioned throughput reservations, agent orchestration runtime, observability and trace storage, safety and guardrail services, identity and secrets infrastructure, network egress, and human-in-the-loop labor. Security controls are not free: they consume services and engineering capacity and should appear in the allocation model rather than as unexplained platform overhead.
Multi-step agents multiply inference and tool cost relative to single-shot completion. Unbounded loops, oversized contexts, and unconstrained tool retries are the main sources of cost volatility. Caching embeddings, retrieval results, and deterministic sub-chain outputs (where freshness and privacy policy allow) reduces repeated spend without changing the success definition.
| Category | Examples | Metering note |
|---|---|---|
| Direct | LLM tokens, embeddings, retrieval queries, tool and API charges, multimodal units | Map to rate cards; meter per request and per step |
| Indirect / shared | Provisioned capacity, orchestration runtime, observability, guardrails, identity, egress, human labor | Allocate by published keys; refresh with invoice cadence |
| Volatility drivers | Multi-step loops, oversized context, unconstrained retries, high-cost tools | Bound with budgets, max steps/tokens, and tool brokers |
Define, validate, and instrument a successful resolution
A successful resolution is not the same as a model completing without error. Publish resolution states and business success criteria per major workflow, including human-escalation paths. Validate outcomes against task completion, quality, and policy checks so cheap but non-compliant or factually wrong answers are not counted as success. Policy-blocked outcomes should remain visible in state machines so security and spend controls are not misread as model failures.
Telemetry, attribution, and reporting dimensions
Cost allocation fails without hop-level instrumentation. Correlate a gateway request ID through planner or agent steps, each model invocation with tokens in and out, each retrieval call, and each tool invocation. OpenTelemetry GenAI semantic conventions define standard span and metric attributes for generative AI client operations, including model, token usage, and operation type, which supports consistent cost and performance attribution across stacks.
Separate metering dimensions for application, agent and version, model or endpoint, user or tenant, tool name, workflow or use case, and environment. Those dimensions enable chargeback, optimization by the highest-cost slices, and joins from runtime telemetry to cloud invoices. Agent multi-step traces are the join key between operational events and unit economic reports.
Publish how shared platform costs are allocated and how often unit-economic dashboards are reconciled to actual billed spend. Without reconciliation, optimization targets artificial costs and governance loses credibility with finance.
Governance and operating thresholds
Map AI unit-economic controls to organizational risk management. Measure cost and quality together; manage runtime limits and human oversight as paired controls. Resolution that is cheap but non-compliant or factually wrong is not success.
Enterprise architects should require a written cost model, dual validation of successful resolution, attribution dimensions that join to invoices, and runtime enforcement for budgets and tools. These requirements keep agents inside cost and risk thresholds as usage scales and as workflows gain more tool authority.
Runtime governance and security platforms can enforce policy, permissions, monitoring, and audit logging at agent runtime. Use them to operationalize least privilege, tool approval workflows, and spend-related controls alongside architectural metering, not as a substitute for a clear unit-economics definition.
Architect checklist for AI unit economics
- Publish inclusion rules for direct, indirect, and shared costs and name allocation keys and refresh cadence.
- Define resolution states and business success criteria per major workflow, including human-escalation paths.
- Instrument every hop with correlated traces and GenAI-aligned attributes for tokens, models, and tools.
- Report cost per query and cost per successful resolution by application, agent, model, tool, tenant, and workflow.
- Enforce budgets, max steps/tokens, tool allow lists, and step-up approval for high-cost or high-risk actions.
- Reconcile unit dashboards to billed spend and gate model, prompt, and tool changes with canary cost checks.
Runtime controls that reduce cost volatility
Pair metering with enforcement so unit costs stay predictable as agents gain tools and traffic grows.
Budgets and quotas
Set per-tenant and per-agent budgets, max steps, max tokens, and tool-call quotas. On breach, alert and degrade gracefully: smaller model, read-only tools, or human handoff rather than unbounded retry.
Model routing
Route classification and extraction to cheaper or smaller models; reserve larger models for hard reasoning. Constrain temperature and max output length by use case to limit unnecessary token growth.
Tool brokers
Isolate high-cost tools behind brokers that enforce authorization, budgets, timeouts, idempotency, and payload size limits before invocation.
Least privilege
Apply least-privilege tool access and secrets hygiene. Prohibit unconstrained shell, payments, or production-write tools without step-up approval.
Change control with cost gates
Treat model/version, prompt, and tool schema changes as cost-affecting releases. Use canary cost gates before full rollout when unit cost or resolution rate may shift.
Auditability
Retain auditable logs of model, retrieval, and tool activity sufficient for cost dispute, security investigation, and regulatory evidence without over-retaining sensitive content.
Operate agents inside cost and risk thresholds
If you are instrumenting agent workflows and need runtime policy, least-privilege tool control, and audit-ready monitoring, talk with the team about runtime governance for enterprise AI agents.
Request a Demo