Runtime Signals That Should Trigger an Out-of-Cycle Review of a Bank AI Agent

What current guidance says about review timing

Policies that define review triggers should cite current guidance. The OCC, Federal Reserve Board and FDIC issued revised interagency model risk management guidance, published as OCC Bulletin 2026-13 on April 17, 2026. It sets out a risk-based approach and rescinds prior model risk management guidance. OCC Bulletin 2011-12 is marked rescinded and replaced by OCC 2026-13. Any trigger policy that still cites the 2011 bulletin, including policies that cite it alongside SR 11-7, should be updated.

The revised guidance describes generative and agentic AI models as novel and rapidly evolving. The Federal Reserve version also states that its principles apply to traditional statistical and quantitative models and to non-generative, non-agentic AI models. A bank should therefore document how it applies those principles to its own agents, using the guidance's risk-based approach. The guidance says practices suited to one bank may be unsuitable for another, and it covers validation and monitoring, including outcomes analysis, as well as clear policies, roles and responsibilities.

The retired 2011 text called for periodic review at least annually, but more frequently if warranted. That text is no longer current, but the design question it raised remains: what evidence shows that review is warranted? As of November 2026, the OCC has also published a supervisory framework for bank use of what it calls Super Intelligence. It states that review triggers, evidence and procedures may be defined, and it refers to thresholds that trigger review, re-validation, reduced autonomy or suspension. Its language is permissive and its terminology differs from "AI agent", so it is best read as a design reference, not a checklist.

Six runtime signal categories

Organize signals by what is observable in the execution path. Each category below can be seen at the level of an individual interaction or tool call. Published observability guidance names drift in output patterns, sudden changes in refusal rates and anomalous tool call sequences as indicators that behavior has shifted. Treating spikes in blocked, redacted or escalated actions as changed-risk indicators is a design inference built on that foundation, not a documented standard. Make that reasoning explicit in your control design.

• Spikes in blocked or redacted actions: A sustained rise against baseline suggests the agent is attempting actions policy did not anticipate, or that inputs have changed.
• Repeated escalations for human approval: Recurring escalations on the same action type indicate the agent is operating at the edge of its approved scope.
• New or unapproved tool and MCP server calls: Any invocation outside the approved tool and action boundary changes what the agent can do and should be reviewed.
• Sensitive data exposure attempts: Attempts to retrieve, emit or transmit data the agent is not permitted to handle, whether blocked or redacted.
• Policy violations: Repeated or severe violations of a specific policy, as distinct from isolated events handled by enforcement.
• Change in the underlying model: A new model version behind the agent can alter behavior without any change to prompts or tools.

Defining thresholds, severity tiers and ownership

No supervisory source supplies numeric thresholds, so thresholds are a bank design choice that must be defensible to an examiner. Establish a baseline for each signal before setting a limit. Observability guidance describes behavior baselining and anomaly detection across output distributions and tool call sequences, and the same approach applies to policy decision rates. Some signals need no statistical baseline. A call to an unapproved MCP server is binary and can trigger review on first occurrence.

Tie severity tiers to graduated responses. The OCC framework names review, re-validation, reduced autonomy and suspension as consequences of crossing thresholds. A reasonable pattern is that a minor rate deviation prompts owner review, a repeated or cross-signal deviation prompts formal out-of-cycle review, and a severe event such as a sensitive data exposure attempt prompts reduced autonomy or suspension pending review. Scale control intensity to the consequences of agent failure or misuse, in line with the guidance's risk-based approach.

Assign each signal a named owner and link it to the model inventory, so the record shows who watches the signal, who can reduce autonomy and who approves a return to normal operation. Also monitor the escalation process itself. The OCC framework notes that high volumes or bulk approval can degrade review quality and cause alert fatigue. A rising escalation count is therefore both a risk signal and a reviewer-capacity signal, and the threshold should account for both. Record changes to thresholds as well, since a threshold quietly loosened is itself a material change.

The enforcement evidence a review should draw on

When an examiner asks what the agent was permitted to do and what it actually did in the interim, the review file needs two things: the policy in force at a given time, and the record of each decision made under it. Published agentic evidence guidance frames the evidence model around agent identity, execution context, tool interactions, authorization, policy decisions and data flows. Practical review evidence per call therefore includes the agent identity, the tool or MCP server invoked, the parameters, the policy applied, the decision (allow, redact, block or escalate), any approver, and the identifier of the model version that served the call. Recording the model version is what makes a model change detectable as a signal. Audit records should be tamper-resistant and correlated to identities, approvals and tool usage.

This is where enforcement location matters. Trussed enforces policy in the execution path across models, agents, tools and MCP servers, covering allow, redact, block, rate limit and escalate for human approval. Because the decision is enforced at runtime, not reviewed after the fact, the audit-ready evidence is generated by enforcement itself rather than reconstructed afterward. A reviewer can then compare permitted behavior with actual behavior for a defined interim period using records produced at the point of decision.

What runtime enforcement data cannot show

Enforcement records have a defined scope: individual interactions and tool calls. Published guidance cautions that signals alone do not establish whether an action was properly authorized, what context influenced it, or whether the records are complete and trustworthy. A review decision should keep enforcement evidence separate from correlated workflow traces and say plainly where each applies.

Trussed does not provide end-to-end tracing of agentic workflows, transaction-level governance across tasks and agents, or model lineage. Enforcement records can show that the model version changed and what the agent did afterward. They cannot show how that model was built or trained. Where a review depends on those items, the bank needs to source them separately and say so in the review file.

• Workflow context: Reconstructing a cascade of actions across agents requires correlated traces beyond per-call records.
• Transaction-level outcomes: Whether a business transaction was handled correctly sits outside per-call enforcement.
• Model lineage: Enforcement can flag a version change but does not document how the model was developed.
• Record completeness: Completeness and trustworthiness of the audit trail need their own controls and periodic checks.