Compliance Guide
Reliance on Vendor Model Documentation in Bank Model Risk Management
Under SR 11-7 and OCC 2011-12, using a vendor or third-party model does not reduce a bank's obligation to independently validate and monitor it. Vendor documentation is a required input to that process, not a substitute for it.
Where documentation is incomplete, proprietary, or silent on runtime behavior, as is common with AI and agentic models, banks are expected to apply compensating controls such as benchmarking, outcomes analysis, and ongoing monitoring, and to extend those principles with runtime observability where static documentation cannot describe actual model behavior.
What SR 11-7 and OCC 2011-12 require for vendor models
SR 11-7 is explicit that the use of a vendor or third-party model does not eliminate a bank's responsibility for model validation. Vendor models must be validated to a standard consistent with models developed internally. OCC Bulletin 2011-12 adopts this guidance for OCC-regulated institutions, applying the same expectations to national banks and federal savings associations that use externally sourced models.
SR 11-7 further specifies that banks should obtain and review available vendor testing results and related information to understand model methodology and limitations. Where that information is not available, the guidance treats the gap as a documented limitation the bank must actively manage, not a condition that excuses further review. Ongoing monitoring is required for all models, including vendor models, to confirm they continue to perform as intended as products, exposures, and market conditions change.
Notably, SR 11-7 defines model risk management scope broadly, covering any quantitative method or system that processes input data into outputs, without carving out an exception based on model type or vendor status. This means the same expectations apply whether the model is a traditional statistical model or an AI system supplied by a third party.
Regulatory basis for vendor model reliance
Four anchors shape how banks should treat vendor and AI model documentation under current supervisory expectations.
SR 11-7 (2011)
Requires vendor models to be validated to the same standard as internally developed models.
OCC 2011-12
Applies SR 11-7 model risk expectations to national banks and federal savings associations.
2023 Interagency Guidance
Requires continuous oversight of third-party relationships across the full lifecycle.
AI and agentic models
Introduce runtime, tool-calling behavior that predates existing guidance and is not addressed by static documentation.
Why vendor documentation alone falls short
Vendor documentation typically describes design intent and validation results captured at a point in time. It is a useful starting point, but SR 11-7 anticipates that vendors may be unwilling to disclose full model methodology to protect proprietary interests. Where disclosure is limited, the guidance expects banks to demonstrate compensating controls, including sensitivity analysis, benchmarking against alternative approaches, and outcomes analysis based on actual performance.
These techniques generally assume a relatively stable relationship between inputs and outputs. That assumption holds reasonably well for many traditional statistical or rules-based models, where behavior can be characterized and re-tested against a fixed specification. It holds less well when the underlying model changes on a rolling basis or produces outputs that are sensitive to context in ways the original documentation did not anticipate.
The documentation gap itself is not a violation of guidance. Failing to apply compensating controls in response to that gap is.
AI and agentic models compound the documentation gap
The gap between documentation and independent understanding becomes more pronounced with AI and machine learning vendor models. These systems can produce outputs that vary with prompts, context, and external data at runtime, which means the actual decision paths and tool-call sequences executed in production may not match what static documentation describes.
Vendors frequently limit disclosure of algorithm internals, training data composition, or model weights, which further constrains the depth of independent technical review a bank can perform. SR 11-7's expectation that model users and validators understand a model's limitations and assumptions is harder to satisfy when the internal logic is not observable from documentation alone.
It is also worth noting directly that SR 11-7 was issued in 2011 and does not reference generative AI, machine learning agents, or tool-calling behavior. Its application to these systems is an extension of general validation principles rather than an explicit rule, and no agency bulletin currently specifies a defined documentation adequacy threshold for AI vendor models. This leaves supervisory interpretation, applied against existing principles, as the practical standard banks must work against.
Where static review works, and where it does not
| Dimension | Traditional vendor models | AI and agentic vendor models |
|---|---|---|
| Primary evidence | Design docs, point-in-time test results, methodology summaries | Same inputs, plus production behavior that docs cannot fully describe |
| Input–output stability | Often stable enough for sensitivity analysis and benchmarking | Outputs can shift with prompts, context, and external tools |
| Proprietary limits | Partial methodology disclosure is common and anticipated | Weights, training data, and internals are often unavailable |
| Compensating focus | Benchmarking, outcomes analysis, ongoing monitoring | Those controls plus runtime logging and behavior capture |
Compensating controls when vendor documentation is incomplete
When vendor disclosure does not support independent understanding on its own, banks should build a deliberate control set rather than treating documentation gaps as residual acceptance criteria.
- Obtain and review all available vendor testing results, methodology summaries, and known model limitations as part of validation.
- Apply sensitivity analysis, benchmarking, and outcomes analysis where vendor disclosure does not fully support independent understanding of the model.
- Establish ongoing monitoring that tracks vendor model performance over time and triggers review when degradation or drift is detected.
- Maintain a centralized inventory of vendor and AI models with risk tiers that account for both financial materiality and degree of dynamic or agentic behavior.
- For AI or agentic vendor systems, implement runtime logging and behavior capture to document actual outputs and tool interactions that static documentation cannot describe.
Governance accountability cannot be outsourced
The 2023 Interagency Guidance on Third-Party Relationships, issued jointly by the OCC, Federal Reserve, and FDIC, reinforces the same principle from a broader third-party risk perspective. It describes a risk management lifecycle covering planning, due diligence, contract negotiation, ongoing monitoring, and termination, and it supersedes prior agency-specific guidance such as OCC Bulletin 2013-29.
The guidance states plainly that banks retain ultimate responsibility for activities performed through third-party relationships and cannot outsource that responsibility, including compliance and risk management obligations. Read together with SR 11-7, this means boards and senior management remain accountable for ensuring the model risk management framework addresses vendor and AI models with governance proportional to their risk, regardless of how limited the vendor's documentation may be.
Contractual provisions that secure ongoing access to vendor testing results, methodology summaries, and advance notice of model changes are one practical way to operationalize this accountability during due diligence and throughout the relationship, rather than only at contract initiation.
Common questions on vendor model documentation
Does SR 11-7 specifically address AI or machine learning vendor models?
No. SR 11-7 was issued in 2011 and does not reference generative AI, machine learning agents, or tool-calling behavior. Its scope is defined broadly to cover quantitative methods and systems that process input into output, so its principles apply to AI models by extension rather than by explicit rule.
Is there a defined adequacy threshold for vendor AI model documentation?
No identified regulatory bulletin specifies a fixed documentation adequacy threshold for AI vendor models. Guidance remains principles-based, which leaves supervisory interpretation, applied against SR 11-7 and third-party risk principles, as the practical standard.
What contractual provisions help address vendor documentation gaps?
Banks can negotiate contractual rights to obtain vendor testing results, methodology summaries, and advance notice of model changes as part of due diligence, consistent with the lifecycle approach described in the 2023 Interagency Guidance on Third-Party Relationships.
How should agentic AI runtime behavior be captured for validation purposes?
Because static documentation cannot describe execution paths that vary at runtime, banks should implement logging and output-capture mechanisms that record actual tool calls and decision sequences, providing an independently reviewable record to supplement vendor documentation.
Assess runtime oversight for vendor and AI models
Static vendor documentation cannot describe how an AI agent behaves in production. Runtime governance provides the observability needed to complement existing validation controls.
Explore Runtime Governance