Building a Monthly AI Agent Ongoing Monitoring Report Under the Revised Model Risk Guidance

What changed in the guidance, and what it means for the report

On 17 April 2026 the OCC issued Bulletin 2026-13, and the Federal Reserve issued SR 26-2, transmitting revised interagency model risk management guidance developed with the FDIC. The agencies describe it as clarifying model risk management principles and setting out a risk-based approach. It rescinds prior model risk management guidance and other issuances. Programs originally built around SR 11-7 and OCC 2011-12 should be re-baselined against the revised text rather than the earlier documents.

The revised guidance keeps the core expectations a monitoring report has to satisfy. Ongoing monitoring evaluates whether a model is performing as expected given changes in products, exposures, activities, clients and data. Outcomes analysis includes reports produced as part of ongoing monitoring. Governance should name who is responsible for monitoring activities, and documentation should be adequate to support the framework. The content of monitoring reports depends on the nature of the model and its materiality.

The OCC bulletin also states that generative and agentic AI models are novel and rapidly evolving. It does not define agent-specific metrics or thresholds. Everything below that goes beyond the guidance text is a proposed report design. Your committee should adopt it as institutional policy, calibrate it, and record the rationale.

The six sections of the monthly report

Reports built for statistical models center on performance, drift and back-testing. An agent acts through tools under delegated authority, so the report also has to show what the agent was permitted to do, what it attempted, and who intervened. The sections below are a proposed structure. Scale depth and cadence to each agent's materiality and document why.

• 1. Inventory and change summary: Agents in scope, business owner, in-house or vendor-supplied, tools and data each may reach, and any change in permitted behavior since last month. Evidence: inventory register, change tickets, policy version history.
• 2. Policy enforcement outcomes: Counts and rates of allow, redact, block, rate limit and escalate decisions per agent and per tool, with month-over-month comparison. Evidence: runtime enforcement records.
• 3. Human approval and overrides: Escalations raised, approved, denied or expired, time to decision, and approver identity. Evidence: approval records tied to the originating tool call.
• 4. Incidents and issues: Items opened, closed and aged, with root cause, remediation owner and response taken. Evidence: issue management system cross-referenced to enforcement records.
• 5. Evidence completeness: Share of in-scope agents producing records, known gaps, and retention coverage against the committee's stated period.
• 6. Escalation triggers and decisions: Which thresholds were breached, what the committee decided, and who owns follow-up.

Metrics, thresholds and escalation triggers

Choose metrics that distinguish individual agent actions from model-level performance. Useful examples include the block rate and escalation rate per agent, redaction events by data category, the proportion of tool calls hitting each policy outcome, approval and denial rates, time to approval, and the count of changes to permitted tools. Pair these with the conventional model measures where the underlying model is also in scope, but keep the two views separate so a stable accuracy number does not mask a change in behavior.

Thresholds are an institutional decision. A workable approach is to run an initial period to establish a baseline per agent, then set bands that trigger review when a metric moves outside them. Separate those band-based triggers from absolute triggers that need no baseline. Examples of absolute triggers are any expansion of tool access with no approval record, any action executed without a required human approval, and any gap in evidence for an in-scope agent.

Watch both directions. A sudden drop in blocks can mean a policy was loosened or the agent's tool access widened. An approval rate near 100 percent can mean reviewers are not examining requests. Longitudinal monitoring may need extra logging and longer observation windows to capture degradation that is not immediately obvious, so size retention for multi-month trend comparison, not just the current month.

Evidence attributes an examiner will look for

No guidance specifies the fields a runtime record must hold, so treat the following as a sensible design baseline. Each record should carry a timestamp, the agent and tool involved, the policy and policy version applied, the action taken (allow, redact, block, rate limit or escalate), and the reason or rule that fired. Escalated actions should add the approver's identity, the decision and the decision time. The retention period should be explicit and tied to the committee's stated policy, with a way to show that records were not altered after the fact.

Overrides deserve particular care. The original SR 11-7 attachment said ongoing monitoring should include analysis of overrides with appropriate documentation. Human-approval outcomes are the closest analogue for agents, so the report should show the pattern of approvals and denials, not just their totals.

A reviewer should be able to pick a figure in the report and trace it to the underlying records. If a metric cannot be traced that way, remove it or flag it as unsupported.

What runtime enforcement evidence can and cannot show

Enforcement in the execution path produces a record at the moment a decision is made. That makes it a durable data source for sections two and three: what the agent attempted, what policy decided, and who approved. Because the record is generated by the decision itself and not reconstructed later, it holds up well when examiners ask what an agent was allowed to do and what it did.

It has limits, and the report should state them as a scope boundary. Runtime enforcement evidence covers individual interactions and tool calls. It does not establish model lineage, dataset provenance or the end-to-end trace of a multi-step agentic workflow. Those need other sources, such as the model inventory, validation files and application-level tracing. A report that presents interaction-level evidence as proof of the whole workflow will not survive a validator's questions.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with allow, redact, block, rate limit and escalate outcomes, and generates audit-ready evidence mapped to NIST and other frameworks. Its deepest regulated sector is insurance. The same approach to evidence applies to any institution that wants records produced at enforcement time.

Mapping the report to frameworks and keeping it defensible

Map each section to a recognized framework so the committee package is easy to review. NIST AI RMF MANAGE 4.1 covers implemented post-deployment monitoring plans. The GenAI Profile (NIST AI 600-1) action MG-4.2-001 calls for regular monitoring and published reports covering performance, feedback received and improvements made. GOVERN 1.5 covers ongoing monitoring and periodic review of the risk management process itself. NIST also describes tracking incidents or errors and their management during operations, which corresponds to the incident section above.

For defensibility, name the owner of each section, record the cadence rationale, and keep the committee's decisions and follow-up actions with the report. Have internal audit assess the monitoring framework, and review the report's own design periodically. Use the revised SR 26-2 and OCC Bulletin 2026-13 text as the reference point, and include vendor-supplied agents in the inventory because the guidance covers vendor and third-party products.

Start with a pilot on the highest-materiality agents, produce one report, and have a validator try to trace five figures back to records. The gaps that exercise exposes are the work list for the next cycle.