Enterprise AI Operating Model KPIs for Banks Under the Revised Model Risk Guidance
A bank's AI operating model is working when measurable indicators show that policy is enforced at runtime, not reviewed after the fact. Track five groups: policy coverage, enforcement outcomes, human escalation, evidence completeness and operational reliability. Each KPI needs a definition, a calculation, a data source and the examiner or model risk question it answers. Adoption and productivity figures show use, not control.
What the runtime layer must capture to support these KPIs
- 1
Per-event record
Identifier, timestamp, actor, target model or tool, policy decision and policy version.
- 2
Approval record
Approver identity, decision, timestamp and time to resolution.
- 3
Retention and residency
Configurable retention and data residency so evidence outlasts review cycles.
Evaluating whether your controls can produce this data
- Can the runtime layer report, per interaction and per tool call, which policy applied and what decision was made?
- Is the KPI denominator established from an inventory, with unmanaged and third-party AI entry points detected?
- Can high-risk actions be routed to human approval, with approver identity, timestamp and resolution time recorded?
- Can evidence for a specific sampled interaction be retrieved on request, and with what latency and retention?
- Are policy changes versioned so that past enforcement results remain interpretable?
- Are thresholds set by risk tier, with named owners and an independent reviewer?
What the KPIs have to prove, and what changed in the guidance
Many banks measure AI adoption and productivity well. Fewer can show that the operating model is controlled and defensible under examiner scrutiny. The KPIs below are built for that second question.
The reference point has moved. OCC Bulletin 2026-13, dated April 17, 2026, announces updated interagency model risk management guidance from the OCC, the Federal Reserve Board and the FDIC, issued as SR 26-2 by the Federal Reserve. The bulletin states that it rescinds prior model risk management guidance, and its footnote identifies OCC Bulletin 2011-12 and SR 11-7 as the prior issuances. As of November 2026, risk leaders should anchor KPI design to the revised guidance and treat SR 11-7 as historical context.
The revised guidance is risk-based and tailored to a bank's model risk profile, size and complexity. It describes ongoing monitoring as evaluating whether a model performs as expected given changes in products, exposures, activities, clients and data. It also highlights outcomes analysis and governance and controls, including clear policies and defined roles. NIST AI RMF points the same way, with ongoing monitoring, tracking of incidents and errors, and periodic evaluation of an organization's indicators and measurements. The KPI definitions in this guide are a proposed method, not a prescribed formula from either source.
Separating outcome measures from vanity measures
A vanity measure describes activity. An outcome measure describes whether a control operated. Active users, prompts per day, code suggestions accepted and hours saved are useful to the business, but none of them answers whether a policy was applied to a given action or what happened when it was.
A practical test: if the number can rise while the bank's control position gets worse, it is a vanity measure for risk purposes. Total AI interactions can double with half of them outside any enforced policy. A block rate can fall because controls stopped working, not because behavior improved. Pair every volume figure with a coverage denominator, and read enforcement rates against a stated expectation for each use case and risk tier.
Thresholds should scale with risk. Higher-risk uses warrant tighter coverage, escalation and evidence targets than low-risk internal assistants, consistent with the risk-based approach in the revised guidance.
Policy coverage and enforcement outcome KPIs
These KPIs show whether policy is applied in the execution path and what it does when applied. Each is calculated per interaction and per tool call, then rolled up by use case, business line and risk tier.
- Policy coverage rate: Interactions and tool calls evaluated by an enforced policy, divided by all interactions and tool calls in scope. Data source: runtime decision records compared with the AI inventory. Question answered: is policy enforced at runtime for the AI actually in use, or only documented?
- Entry-point coverage: Known AI entry points (models, agents, tools, coding assistants, third-party AI tools) under enforcement, divided by all known entry points, reported with a count of unmanaged entry points detected. Data source: inventory and discovery. Question answered: how is the denominator established, and is it complete?
- Enforcement decision distribution: Counts and rates of allow, redact, block, rate limit and escalate decisions by policy, use case and risk tier, trended over time. Data source: per-event policy decisions. Question answered: do controls act, and are shifts in rates explained?
- Exception rate: Actions permitted under an approved exception, divided by actions that would otherwise have been blocked or escalated. Data source: decision records and the exception register. Question answered: are exceptions bounded, approved and reviewed?
Escalation, evidence and reliability KPIs
These KPIs show whether humans are in the loop where policy requires it, whether the record supports review later, and whether the enforcement layer itself can be relied on.
- High-risk routing rate: High-risk actions routed to human approval, divided by actions that policy classifies as high risk. Data source: escalation records and policy classification. Question answered: do high-risk actions reach a person before they execute?
- Escalation resolution: Median and tail time from escalation to decision, plus the approve and deny split, with approver identity and timestamp recorded. Data source: approval workflow records. Question answered: is human review timely, attributable and meaningful?
- Evidence completeness: Events with every required field present (identifier, timestamp, policy version, decision, and approver where relevant), divided by all events in scope. Data source: evidence store checks. Question answered: can any action be reconstructed without gaps?
- Evidence retrieval success: Sampled events retrieved in full within a target time, divided by events sampled. Data source: periodic retrieval exercises. Question answered: can evidence be produced on request, not merely stored?
- Enforcement availability: Time the enforcement point is available, plus counts of failure events and how each was handled. Data source: service telemetry. Question answered: what happens to AI traffic when the control fails?
- Policy change traceability: Policy changes with a recorded version, owner and approval, divided by all policy changes. Data source: policy repository. Question answered: can historical enforcement results still be interpreted?
Five KPI groups for AI control effectiveness
Policy coverage
Share of AI interactions and tool calls under enforced policy.
Enforcement outcomes
What was allowed, redacted, blocked, rate limited or escalated.
Human escalation
High-risk actions routed to approval, and how they resolved.
Evidence completeness
Records that are complete and retrievable on request.
Operational reliability
Whether enforcement itself stays available and traceable.
Check whether your runtime controls can produce these KPIs
See how policy enforced in the execution path generates per-interaction and per-tool-call evidence that supports coverage, enforcement and escalation measures.
See How Evidence Is Produced