Implementation Guide | Financial Services

    KYC Agent Override Rate Measurement Under the 2026 Model Risk Management Guidance

    Define the override rate as the share of eligible agent outputs that analysts change or reject, and write down the numerator, denominator and exclusions. Never present it alone. Segment it by risk tier, alert type and agent confidence, pair it with review-effort signals and known-answer cases, set institution-defined thresholds for low or uniform behavior, and retain tamper-resistant records of agent output, analyst action, rationale and escalation approval.

    Records to retain for examiner requests

    • Agent output and confidence score as presented to the analyst, with the agent version.
    • Analyst action, including override, partial edit or approval, with timestamp and identity.
    • Written rationale as submitted, preserved without later edits.
    • Review-effort signals: time on task and evidence opened.
    • Escalation approvals for agent actions and tool calls, with approver and outcome.
    • Metric specification versions, thresholds, breach reviews and known-answer results.

    What changed in model risk management guidance

    On April 17, 2026, the OCC, in coordination with the Federal Reserve Board and the FDIC, issued updated model risk management guidance. It serves in part to rescind prior model risk management guidance and other issuances. As of November 2026, policies and validation documents that cite earlier guidance should be reconciled to the current issuance.

    The guidance is risk-based and tailored. It says practices should be commensurate with a banking organization's size, complexity and extent of model use, and that practices effective for one institution may be inappropriate for another with a different risk profile or model purpose. It addresses model testing, validation and monitoring, including outcomes analysis, and governance and controls, including clear roles and responsibilities. The agencies also describe generative and agentic AI models as novel and rapidly evolving, and planned a request for information on model risk management that considers AI use.

    The practical consequence is that no regulator-prescribed override metric exists. The institution has to design the measure, justify why it is proportionate to its KYC agent deployment, and keep evidence that it works. A useful parallel is OCC fraud risk guidance, which treats override monitoring as part of internal control. That context is fraud, not KYC, but the control logic carries over: overrides are a signal that needs surveillance, not just a statistic.

    Define the override rate and its denominator

    A low override rate can mean an accurate agent or analysts approving without scrutiny. The number cannot separate the two, so its value depends on how precisely it is defined. Write a metric specification before collecting data, and have model risk management validate it.

    The denominator matters most. If it includes every agent output, including low-risk auto-cleared items that no analyst touched, the rate will look artificially low. A defensible denominator is the set of agent outputs that were routed to a human reviewer and were eligible for change.

    • Numerator: state what counts as an override, such as a reversed disposition, a changed risk rating or a rejected recommended action.
    • Partial edits: decide whether a corrected field or amended narrative counts, and report these separately rather than discarding them.
    • Denominator: reviewed agent outputs only, with the routing rule that sent them to a human documented.
    • Exclusions: list cases removed, such as system errors or duplicates, and report exclusion counts each period.
    • Unit of analysis: fix whether you count per case, per alert or per agent recommendation, and keep it consistent.

    Segment, baseline and set thresholds

    An aggregate rate hides the places where scrutiny matters most. Segment by customer risk tier, alert type and agent confidence level. In a healthy process, overrides would be expected to vary across these segments: more disagreement on high-risk customers or low-confidence outputs, less on routine, high-confidence ones. A rate that is flat across every segment, or near zero in high-risk tiers, deserves more attention than a low overall figure. Also segment by analyst and team, since uniform behavior by a single reviewer is a distinct signal.

    Run an initial baseline period before setting thresholds. No standard benchmark exists for acceptable override rates, so thresholds are institution-defined and should be validated internally and justified as commensurate with the size, complexity and extent of use of the agent. Define triggers on both ends: overrides below a floor in a segment, variance across segments or analysts below a minimum, and sudden shifts after an agent or prompt change. Each breach should have a named reviewer, a required response time and a recorded outcome. A breach is a prompt for investigation, not a finding of misconduct.

    Pair override rate with indicators of genuine review

    Automation bias, the tendency to accept a machine recommendation without independent checking, is the mechanism behind rubber-stamping. Override rate measures its output only indirectly, so complementary indicators should measure the effort and quality of review. These are design choices, and none proves meaningful review on its own. Used together, they make a stronger case than any single measure.

    • Seeded or known-answer cases: Insert cases with a known correct outcome, including deliberately flawed agent outputs, under documented governance. The rate at which analysts catch them is a direct test of scrutiny.
    • Review-time distributions: Track time on task by segment. A tight cluster of very short reviews on complex cases is more informative than the mean.
    • Evidence opened: Record which source documents, screening hits and supporting data the analyst viewed before deciding.
    • Rationale quality: Sample written rationales and score whether they reference case-specific facts rather than restating the agent's output.
    • Disposition changes after QA: Compare analyst decisions with independent quality assurance outcomes to see whether approvals hold up.

    Where runtime policy enforcement supports the record

    Examiners and model risk management teams will ask for evidence, not assertions. Evidence is most reliable when it is generated at the point of action rather than reconstructed from analyst-editable case notes.

    Trussed enforces policy in the execution path for models, agents and tools, and can escalate individual agent actions and tool calls for human approval. Enforced at runtime, not reviewed after the fact, each escalation produces audit-ready evidence that a human decision point occurred and what was decided. This applies to individual interactions and tool calls. Trussed does not provide transaction-level governance or end-to-end tracing of an agentic workflow, so case-level context must come from your case management system.

    Escalation records do not by themselves prove the review was meaningful. Approval after a two-second look still produces an approval record. Treat them as one input, and join them with the time-on-task, evidence-opened and known-answer indicators above. The same measures can be applied whatever tooling you use.

    From override rate to examiner evidence

    Define

    Numerator, denominator, partial edits and exclusions in a written metric specification.

    Segment

    Risk tier, alert type and agent confidence, watching for uniform behavior.

    Corroborate

    Review time, evidence opened, known-answer cases and rationale quality.

    Retain

    Agent output, analyst action, rationale and escalation approvals as durable records.

    Show examiners how review was enforced and recorded

    See how runtime policy enforcement, escalation for human approval and audit-ready evidence can support your oversight records.

    See How Evidence Is Produced
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.