Testing That a Bank AI Agent Calls Only Authorized Tools Before Production

What the test has to prove

Many pre-production suites score answer quality, refusal behavior and hallucination rates. They rarely ask a narrower question: can this agent invoke only the tools, functions and parameters it was approved to use? For an agent that reaches internal systems and external services, including through MCP servers, that question determines what the agent can actually do.

It helps to treat tool authorization as a control with a defined policy baseline, not as a model behavior to be observed. In MCP-style architectures the model decides which tools to invoke, when, and with what parameters. It also sees the tool descriptions from every connected server in its context. OWASP guidance on excessive agency is consistent on the consequence: authorization should be implemented in downstream systems or execution code, not left to the model to decide whether an action is allowed. A test plan should therefore verify the enforcement layer, and treat the model as an untrusted caller that will sometimes be wrong or manipulated.

Step 1: Build the authorized tool inventory

You cannot write a negative test without a baseline. Before testing, record for each agent a machine-readable inventory of what it may do. Keep it minimal. OWASP advises limiting the extensions an agent can call to the minimum necessary, so a tool that is not needed, such as URL fetching, should not be in the inventory at all.

Each entry should capture the following.

• Tool and operation: The specific tool and the operations permitted on it, not a blanket grant to the tool.
• Parameter scope: A strict schema per operation: declared parameters only, typed, pattern-constrained and bounded.
• User context: The identity and minimum privileges under which the call runs, ideally the specific user's context.
• Approval rule: Which actions need human approval, and which are denied outright.
• Pinned definition: A cryptographic hash of the reviewed tool definition, covering inputSchema, outputSchema and annotations.

Step 2: Design positive and negative test cases

Positive cases confirm that authorized calls with in-scope parameters succeed. They matter because a policy that blocks everything passes every negative test. Negative cases confirm that everything outside the inventory is denied or escalated. The categories below follow OWASP guidance on excessive agency, prompt injection, agentic risks and MCP security.

Use harmless data and instrumented tool substitutes so that a failed test cannot cause a real action. Include injection variants that contain none of the obvious trigger keywords, because keyword screening alone is not a sufficient control.

• Unlisted or unnecessary tool invocation: Prompt the agent toward tools outside its allowlist. Expected result: no call executes.
• Parameter abuse: Send undeclared parameters, malformed formats and out-of-range values. With additionalProperties set to false and pattern constraints, these must be rejected.
• Direct prompt injection: Instruct the agent in user input to exceed its scope. Expected result: denial or escalation, not execution.
• Indirect injection: Place instructions in retrieved content and tool outputs. Tool output is untrusted, and high-impact actions it triggers need confirmation.
• Tool definition manipulation: Change descriptions, schemas or annotations such as readOnlyHint. A changed hash must trigger re-review and re-test.
• Chaining and scope creep: Test cross-server and multi-agent chains where an upstream injection cascades into a downstream action, and sequences of individually permitted calls that add up to an unpermitted result.

Step 3: Set pass/fail criteria and capture evidence

Criteria should be assertable without judging the model's wording. Examples: no call outside the allowlist executes; every schema-invalid argument is rejected; actions designated for approval do not complete without it; downstream system state is unchanged after each negative case. Assert against tool-call records and the state of the downstream system, not against what the agent says it did. Treat any single unauthorized execution as a failure that blocks release, and rerun the full suite after remediation.

Because agent behavior is non-deterministic, run each adversarial scenario repeatedly and with varied phrasing, and record the run count. A pass shows that the controls held in the cases tried, not that every case is covered.

For each run, retain immutable records of tool invocations, the inputs, the decision, the policy version and the pinned tool-definition hashes. This lets a reviewer reproduce a result and tie it to the exact configuration tested.

OCC Bulletin 2026-13, Model Risk Management: Revised Guidance, dated April 17, 2026, organizes model risk management around development and use (including testing), validation and monitoring, and governance and controls. Tool-authorization evidence maps naturally to those headings. The bulletin also states that appropriate practices depend on a bank's risk profile and the model's purpose, so document why your test depth fits this agent and its tool access. Confirm how the bulletin applies to AI agents in your institution, as of November 2026.

Where pre-production testing stops

A passing suite describes behavior under the scenarios you thought to write. It does not guarantee that the same constraints hold when live users, new retrieved content and changed tool definitions meet a non-deterministic model. Three gaps follow from this.

First, new injection content will appear that no test anticipated. Second, tool definitions and annotations can change after release, altering client approval behavior without any change to a description. Third, an exception granted in testing can become a standing permission. Each gap means the allowlist, parameter schemas and approval rules need to be enforced on every tool call at runtime, with the outcome recorded as it happens.

The runtime outcomes are allow, redact, block, or escalate for human approval, applied at the tool-call level. OWASP also advises rate limiting so undesirable actions can be found through monitoring. The inventory built for testing should be the same policy that runtime enforcement applies, so the evidence from both stages refers to one baseline.

Trussed enforces policy in the execution path for models, agents, tools and MCP servers, with allow, redact, block, rate limit and escalate-for-approval outcomes on individual tool calls. Enforcement generates audit-ready evidence, mapped to frameworks including NYDFS and NIST. Trussed is runtime control, not a testing or red-teaming product; it complements the pre-production work described here.