How to Run a Security Review of a Customer-Hosted AI Control Plane at a Bank Under Current Model Risk Guidance

Start from the standards the review must satisfy

A customer-hosted AI control plane sits in the path between applications, models and tools. A bank therefore reviews it as a third-party component, as a system affecting model risk, and as part of its information security program. Scoping starts by naming the standards and deriving testable control objectives from each.

On model risk, the Fed, OCC and FDIC issued Revised Guidance on Model Risk Management under SR 26-2, which supersedes and replaces an earlier SR letter. As of November 2026, scope the review to the revised guidance, and confirm with your model risk function whether the control plane falls within its inventory. The earlier SR 11-7 attachment frames model risk management as governance and control mechanisms: board and senior management oversight, policies and procedures, and controls and compliance. Those themes translate directly into review evidence: documented ownership, written procedures and demonstrable controls.

On information security, the NYDFS Part 500 Second Amendment requires each covered entity to maintain a cybersecurity program designed to protect the confidentiality, integrity and availability of its information systems and nonpublic information. Your review file should trace to that program. For AI-specific threats, the NIST Generative AI Profile lists backdoors, compromised dependencies, data breaches, eavesdropping, man-in-the-middle attacks, autonomous agents, model theft and AI inference, bypass and extraction. Use that list as a threat checklist. OWASP AISVS offers testable requirements, with Level 3 aimed at regulated industries and high-assurance environments.

Review identity, secrets and data handling

NIST SP 800-239 describes multi-layered security that combines end-to-end encryption, strict identity and access management and input and output token monitoring, and it contrasts zero trust with implicit trust. Apply that posture to three populations: administrators, service identities and policy authors. Confirm least privilege, strong authentication, separation between those who write policy and those who approve it, and a record of every policy change.

For secrets, inventory every credential the component holds: model provider keys, tool credentials and service tokens. Establish where each is stored, how it is rotated, who can read it and whether it is ever written to logs or evidence. For network egress, require an explicit allowlist and verify it from outside the component, not only from its configuration.

Prompts and responses deserve their own data classification. Because responses are logged alongside queries, evidence stores can accumulate nonpublic information. Confirm encryption in transit and at rest, access controls on the evidence store, and whether redaction is applied before content is persisted. These are reviewer-defined criteria, so record the bank's chosen standard for each in the file.

Validate failure modes, bypass paths and drift

The NIST Generative AI Profile calls for verifying that the architecture can monitor outputs and performance and can handle, recover from and repair errors when security anomalies are detected. Turn that into explicit failure tests. Disable the policy engine, its policy store and its identity provider in turn. Record whether traffic is refused or allowed, who is alerted and how recovery is verified. Whether the component should fail closed or open is a bank decision tied to the risk of each use case, but it must be a documented decision, and the observed behavior must match it.

Bypass testing is a network and identity exercise as much as an application one. Attempt to reach each model endpoint and tool from application workloads without traversing the component. OWASP guidance supports adversarial testing, source code review and red teaming, and asks third-party providers for independent audits, penetration tests and code reviews. Request these, and plan to repeat your own tests after upgrades or policy changes.

For policy drift, compare the deployed policy set against the approved baseline and confirm that changes require approval and are attributable. Define review triggers up front, including escalation to human involvement after a control breach or misfire, and have someone independent of the team operating the control assess the results where appropriate.

Assess whether audit evidence will satisfy an examiner

NIST SP 800-239 treats logging users' queries and API calls, together with the corresponding model responses, as a key initial step for detecting suspicious activity. For a bank, the bar is higher: evidence must answer an examiner's question about what the AI was permitted to do and what it did. Assess five properties.

Completeness: every model call and tool call produces a record, including blocked, redacted and escalated actions, and the review reconciles record counts against independent traffic data. Integrity: records cannot be altered or deleted by the operators of the component, and tampering is detectable. Retention: periods are configurable, set by the bank's records policy and enforced. Residency: evidence is stored in the regions the bank requires, including backups. Mapping: each record type ties to a named control objective, so an examiner can move from a requirement to the evidence in one step.

These five properties are reviewer-defined. Document the standard you applied to each and the test that showed it was met. Because the review is not a one-time event, also plan ongoing monitoring and periodic retesting, with metrics on AI security, reliability and robustness reported to senior management.

Document findings in an examiner-ready file

Treat the review file as the deliverable. It should let someone who was not present reconstruct what was examined, how, and with what result.

• State the boundary, the applicable standards as of the review date and the control objectives derived from each.
• Record every test case with its inputs, expected result, observed result and date, and keep repeated-run results together.
• Rate each finding by the control objective it affects, name an owner and set a remediation date.
• Record the model risk management scope decision and the rationale for any item judged out of scope.
• Attach vendor assurance material, such as independent audit or penetration test reports, and note what was independently verified.
• Define the retest trigger for upgrades, policy changes and incidents.

Where a runtime enforcement product fits

The criteria above are vendor-neutral and apply to any customer-hosted component in the AI execution path. As one factual reference point, Trussed is deployed self-managed in the customer cloud and enforces policy in the execution path across models, agents, tools and MCP servers, with actions of allow, redact, block, rate limit and escalate for human approval. It generates audit-ready evidence mapped to frameworks such as NYDFS and NIST, with configurable retention and data residency. Insurance is the regulated financial sector where Trussed is deepest. Treat these as claims to test against the checklist above, not as a substitute for it.