Requiring Human Approval the First Time an LLM Agent Uses a New Tool

Why first use is the right checkpoint

Agents can be configured with, or discover, tools after launch. That includes tools exposed by MCP servers. Once a tool is reachable, the agent can call it with real credentials and data. A static allowlist covers only what was known at review time, and a pre-deployment review cannot cover a server that adds or changes tools later.

First use is a natural decision point because it is the moment a tool moves from unknown to exercised. The MCP specification, as described in OWASP's MCP guidance, says hosts must obtain explicit user consent before invoking any tool and that a human should be able to deny invocations. OWASP's agent guidance goes further on placement: authorization belongs in the execution component, outside the agent's context, and it should fail closed for unknown tools or missing approval requirements. A tool the policy has never seen is treated as high risk by default and requires human approval.

The model deciding to call a tool is not the same as the user being authorized to use it. Approval logic must therefore never depend on claims the model or the tool server supplies.

Define what counts as a new tool

A tool name is not an identity. Names can be shadowed by a malicious server, and descriptions and schemas are part of the agent's context, where they can carry prompt-injection payloads. OWASP also describes the rug-pull pattern: a server behaves safely during review, then changes its tool definitions after approval.

A workable identity for the 'new tool' test combines the server or endpoint, the tool name, the description, the input schema and the server version. Snapshot these at approval time and compare them on each connection or call. Any difference moves the tool back to unseen and requires re-approval. Pinning definitions and monitoring for name shadowing support the same goal.

Treat tool annotations, such as read-only labels, as untrusted unless they come from a trusted server. A read/write label alone also does not establish risk, because sensitive reads, data disclosure and irreversible actions matter too. Use annotations as hints for the approver, not as inputs that skip review.

Choose approval scope and memory

Decide what an approval covers: the tool, the tool plus server version, a specific agent, a specific user, or an environment. Broad scopes reduce prompts but create over-broad approvals. A tool approved once for every agent and environment turns one review into standing access. A practical default is to bind approval to the exact definition snapshot, the agent identity and the environment, and to widen scope only for low-risk tools.

Separate two ideas. First-use approval admits a tool definition into an allowed set. Per-call approval authorizes a particular invocation. OWASP advises against assuming that an available tool has consent for any given invocation. For high-risk calls, make approvals single-use: bound to the actor and exact call, checked and consumed in one atomic step before execution, with new approval required if the target or parameters change. A 'user_confirmed' flag does not meet that bar.

Classify actions by reversibility, for example read-only, reversible, externally reversible and irreversible. Use the class to decide what runs unattended and what needs approval. The class informs risk handling but never grants permission to run a tool on its own. Pair approvals with least-privilege tool allowlists and short-lived, task-scoped credentials so approval is not the only control.

Approver context, denial, timeout and fatigue

Show the approver the full tool call parameters, not just the tool name, along with the tool description, the server, the requesting user and agent, and the data involved. Review descriptions for hidden instructions. An approver who sees only a tool name cannot judge exfiltration through arguments.

Define behavior for every non-approval path before launch. On denial, return a structured refusal so the agent can choose another route and does not retry in a loop. On timeout, fail closed and move the request to expired. Set timeout lengths and approval-memory durations by risk tier, because there is no universal value. On revocation, apply the change at runtime so the next call is blocked. Add circuit breakers for unusual call volume or endpoints the agent has not used before.

Approval volume is a control-design problem. NIST material warns that frequent escalation leads to habitual approval, called consent fatigue. Set risk thresholds that trigger human review, route requests to the right authority, and let low-risk, already-approved definitions proceed. Fewer, better-informed approvals are stronger than many reflexive ones.

Runtime enforcement versus review after the fact

Reviewing tool use afterward tells you what happened. It cannot stop a first call from sending credentials or data to an unreviewed tool. Enforcement in the execution path evaluates the call before it runs, so unknown tools are held, definition changes are caught, and approvals are consumed at the moment of use. Records produced by that enforcement are evidence of a decision, not a reconstruction of one.

When evaluating a solution, ask whether enforcement sits in the execution path and fails closed for unknown tools. Ask whether approvals are bound to actor, call and parameters and are single-use. Ask whether it snapshots definitions and triggers re-approval on drift, whether approvers see full parameters, whether timeout behavior is configurable, and whether it supports risk-threshold escalation to limit fatigue.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers. For individual interactions and tool calls, it can allow, redact, block, rate limit or escalate for human approval, and it generates audit-ready evidence from that enforcement. It is built for regulated enterprises that must show what their AI was allowed to do and what it did.