Ongoing Monitoring for Bank AI Agents Inside the Bank's Own Environment

What supervisors expect from ongoing monitoring

Supervisory material treats ongoing monitoring as an expectation but does not prescribe a method. The interagency guidance on third-party relationships says the agencies are not encouraging any specific approach. Monitoring should fit the risk of each relationship and the bank's size, complexity and risk profile. It can be periodic or continuous, and relationships that support higher-risk activities warrant more comprehensive or more frequent monitoring.

The Federal Reserve's SR 26-2 attachment on model risk management describes ongoing model monitoring as evaluating whether a model is performing as expected given changes in products, exposures, activities, clients and data. For vendor models, it points to ongoing monitoring and outcome analysis, and notes that banks may not receive underlying code or data from the vendor. For an agent that calls tools and touches customer records, that means the bank needs its own observations of behavior.

An OCC supervisory framework document on agent use states that accountability for managing risk generally remains with the bank when vendors are used. It also lists bank or customer information in vendor-hosted environments that are not fully transparent to the bank as a risk. It adds that a bank may define the scope of authority, identity, permitted tool use, data use and escalation limits for provider agents, and may monitor and potentially limit what they do. As of November 2026, these are the anchors for treating customer data residency as an architectural requirement rather than a procurement preference.

The practical implication is that examiners will ask what the agent was allowed to do, what it did, and whether the bank can show it. The bank should own and be able to retrieve that evidence itself.

Signals to capture, and what can stay out of the record

Industry guidance on AI observability describes input monitoring (prompts, retrieved context, tool outputs, conversation history), output monitoring, behavior baselining, trace logging for agentic systems and anomaly detection. It also covers latency and token usage, which can indicate denial-of-service conditions or unusual inference loads. A NIST NCCoE discussion essay, which is not final guidance, lists agent audit-record content as users and other agents, workflow context, tool usage and approval events, and advises preserving records in a tamper-resistant manner where appropriate.

From these, a working trace schema for each interaction includes the prompt, retrieved context, tool call and output, the policy decision, any approval or escalation event, the identity of the user and the agent, and the workflow context. Add latency and token metrics for baselining.

Not every field needs to hold raw customer data. Redaction or hashing before persistence is a design choice for the bank, and no source prescribes it, but it follows the same logic as the advice to keep secrets out of outputs and logs. A common split is to store a hash or redacted form of the payload alongside the decision, the policy version and the identities involved. Where an investigation needs the original, keep it in bank-controlled storage under tighter access, not in a vendor system. Decide field by field and document the reasoning, because examiners will ask why a given record is, or is not, reconstructable.

Retention and residency are bank policy decisions. Set the storage region to match residency requirements, set retention per record class, and test that an export for an examiner request works without vendor involvement.

Three deployment patterns and their data egress

The same monitoring features can be delivered in three ways, and the data flows differ. This taxonomy is a way to structure the evaluation, not a standard. Use it to ask where each signal travels and who controls storage.

• Vendor-hosted SaaS: Prompts, responses and tool-call payloads are sent to the vendor's environment. Residency and retention are set by the vendor's configuration. Highest conflict with the OCC concern about environments not transparent to the bank.
• Vendor-managed control plane, customer-side collectors: Collectors run in the bank's cloud, but management, analytics or metadata may flow to the vendor. Egress depends on what the collectors forward, so each forwarded field needs review.
• Self-managed in the customer cloud: Enforcement, evidence storage and retention all run in the bank's account. The bank controls egress paths and residency. It also takes on operating the deployment.

Implementation sequence

Start with scope and risk, then build outward. Because supervisory expectations tie monitoring depth to risk, the agent inventory and tiering drive every later decision. Include provider and external agents that interact with bank systems, not only agents the bank built.

Tradeoffs are real. Self-managed deployment gives the bank control over data flow but means it owns availability, upgrades and capacity. Enforcing in the execution path adds a dependency, so decide up front what happens if the enforcement component fails, for example fail closed for high-risk tools and fail open only where policy explicitly permits.

• Build the agent inventory and assign risk tiers, including provider agents.
• Define the trace schema and mark each field as raw, redacted, hashed or excluded.
• Deploy enforcement in the execution path within the bank's cloud account and write policies for scope, permitted tools, data use and escalation limits.
• Configure the evidence store with tamper resistance, retention and region, and test export.
• Set baselines and thresholds for latency, token use and behavior, with named human approvers for escalations.
• Feed monitoring outputs into model-risk review of whether each agent still performs as expected.