Enforcing Task-to-Model Mapping at Runtime to Cut LLM Costs

Advisory routing versus enforcement

Most organizations already have a task-to-model mapping somewhere. It lives in a design review, a wiki page or application configuration. Summarization uses a small approved model, classification uses another, and customer-facing generation uses a reviewed model with additional controls. The problem is that nothing checks the mapping when a request is made.

Advisory routing suggests a model, and the application is free to ignore the suggestion. A developer can hard-code a frontier model, point a new feature at a model nobody approved, or call a provider directly. Enforcement is different. The decision is made in the execution path at runtime, not reviewed after the fact, and a non-conforming request does not reach the model unless policy says it may.

This distinction matters for regulated enterprises because the question an auditor asks is not what the design review said. It is which models were permitted for which tasks and what actually ran. Spend usually falls as a side effect, because expensive models stop appearing where a smaller approved model is sufficient. Treat that as a consequence of the policy, not as the reason to build it.

Define the policy and establish task identity

Start with a task inventory. The NIST AI RMF, in MAP 2.1, asks organizations to define the specific tasks and methods an AI system will support, such as classifiers, generative models and recommenders. That inventory becomes the taxonomy for the policy. For each task, list the approved models, the default for models not listed, and the risk tolerance that sets how strict enforcement should be. The framework directs organizations to define risk tolerance explicitly, and that decision is a sensible way to separate tasks needing hard blocks from those where rate limiting or escalation is enough. The structure resembles role based access control: task-scoped model entitlements play the part that roles play for users.

The policy is only as good as the task identity behind it. Four request attributes are commonly available, and each has limits. These limits are design judgment rather than documented benchmarks.

Application identity, such as a service credential issued by the enforcement point, is hard for a caller to forge but is coarse when one application runs several tasks. Caller identity, meaning the user or service principal, supports entitlements per team or role but says little about the task. Tool context, such as which tool or MCP server initiated the call, is precise for agent actions but absent in direct completions. The declared task type, passed as a header or metadata field, is the most expressive signal and the easiest to misstate.

The practical rule is to bind the declared task type to an identity the application cannot change. For example, a credential is entitled to a set of task types, and a declared type outside that set is rejected. Avoid inferring the task from prompt content. Prompt injection, which NIST defines as exploiting the concatenation of untrusted input with a prompt built by a higher-trust party, shows why content is a weak basis for identity.

Choosing where to enforce

Enforcement can sit in several places. The right choice depends on how much bypass resistance you need and how much operational weight you can carry. Many deployments combine more than one layer.

• Inline proxy in the request path: Every model call passes through it, so it can decide before the call and cannot be skipped if direct provider access is closed. It adds a network hop and is a component to run and keep available.
• SDK middleware: Runs in the application process with minimal added latency and is easy to adopt. It depends on every team including it and configuring it correctly, so an application can omit it or call a provider directly.
• Network-level enforcement: Egress controls restrict which provider endpoints workloads can reach. It resists bypass well but sees destinations rather than tasks, so it supports an allowlist of providers or models, not a mapping by task.
• Combined pattern: Close direct egress to model providers and place task-aware policy decisions in a proxy in the path. Use SDK middleware only to attach task metadata to requests.

Decide how violations are handled

Decide the default first. A model that is not on the list for a task should be denied unless policy states otherwise, and that includes newly released models. Default allow turns every new model into an unapproved use until someone notices.

For requests that do violate the mapping, the policy should support a range of responses configured per task. Allow is appropriate in a monitored transition, where you record the violation but let the call proceed. Redact applies when the model is permitted but the payload contains data the task should not send. Block suits tasks with strict tolerance, such as customer-facing generation. Rate limit fits cases where a higher-tier model is allowed for a task but only in bounded volume. Escalate for human approval handles legitimate exceptions, such as a one-off task needing a more capable model, and should record who approved it and when.

These five actions are design guidance rather than a documented standard. Two failure modes recur. Over-broad allowlists, such as every model permitted for every task, satisfy the policy without constraining anything. Declared task types that are never validated allow a caller to claim the most permissive task. Review enforcement outcomes as part of post-deployment monitoring, which the AI RMF treats as ongoing work including incident tracking and model monitoring.

How Trussed enforces policy in the execution path

Trussed is runtime control and evidence infrastructure for regulated enterprises. It enforces policy in the execution path on individual interactions and tool calls across models, agents, tools, MCP servers, AI sites and coding assistants. Actions include allow, redact, block, rate limit and escalate for human approval. A task-to-model rule is applied when the request is made, and over 100 models sit under one policy plane.

Enforcement produces continuous audit-ready evidence, mapped to frameworks including NIST and the EU AI Act. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency, and has run for more than 18 months in continuous production in a regulated enterprise.

The scope is the individual interaction and tool call. Trussed does not provide transaction-level governance across tasks, model lineage, or end-to-end tracing of agent workflows.