SR 26-2 Examiner Readiness Checklist for Generative and Agentic AI

Why a checklist is needed for generative and agentic AI

Examiners ask two questions of any AI deployment: what was the system permitted to do, and what did it actually do. Model risk management practice was designed for statistical models, where pre-deployment validation and periodic review give reasonable assurance because outputs are bounded and repeatable. Generative systems produce open-ended outputs. Agentic systems go further and call tools, so a single interaction can read data, write to systems or trigger a transaction.

For those systems, a validation report and an annual review do not show behavior on a given day. The checklist below converts supervisory expectations into artifacts you can hand over. For each area it names the expectation, the evidence an examiner would plausibly request, and the control or artifact that satisfies it. The evidence lists are planning assumptions, not a quotation of any examiner manual.

NIST AI RMF and its Generative AI Profile, published in July 2024, are useful cross-references for structuring the program. NIST states the AI RMF is intended for voluntary use, so treat it as a framework reference and not as an examination standard.

Scope and inventory: a governance process, not a runtime function

NIST AI RMF GOVERN 1.6 states that mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities. The inventory is therefore an organizational mechanism with an owner and a budget, not something a runtime layer produces on its own.

The NIST Playbook suggests setting a policy that defines which models or systems are inventoried. It prefers inventorying all of them or, at minimum, high-risk systems and systems deployed in high-stakes settings. For generative and agentic AI, the practical decision is whether the inventory stops at models or also lists agents, the tools they can reach and the data those tools touch. An examiner who finds an agent with write access to a system of record that does not appear in the inventory will question the rest of the program.

Document the scoping decision itself. If the inventory covers only high-risk systems, record how that classification was made and who approves changes to it.

What runtime enforcement can evidence, and what it cannot

Runtime enforcement operates at the level of individual interactions and tool calls. At that level it can record the policy decision applied (allow, redact, block, rate limit, or escalate for human approval) and the interaction it applied to. That directly serves areas 3, 4 and part of 5 in the checklist.

Several expectations sit outside that layer and need their own programs. Pre-deployment validation and testing, which NIST treats as its own resource area under testing, evaluation, verification and validation, establish whether a system is fit to deploy. Model lineage records where a model came from and how it changed. Workflow-level tracing reconstructs how a multi-step task moved across agents. Inventory governance and the written policy itself are authored by people, then enforced by runtime controls.

The boundary matters in an examination. If a reader claims runtime records answer a lineage question, the first follow-up exposes the gap. State plainly which artifact answers which question.

Post-deployment monitoring is the closest NIST lifecycle concept to runtime evidence. The NIST Playbook suggests tracking and monitoring negative risks and benefits throughout the AI system lifecycle, including after deployment. Ask whether your monitoring records individual interactions and tool calls or only aggregate metrics, because the answer determines what you can show for a specific event.

Designing evidence that holds up

The examiner-facing value of a record depends on how it was produced. These are design considerations for your own program, not requirements attributed to any regulator.

Completeness: records should cover every interaction and tool call in scope, including those that were blocked or escalated. A record set containing only successful actions misrepresents behavior.

Traceability to policy: each record should identify the policy decision applied. If you can also identify the policy version in force at that moment, you can explain why a past action was allowed under rules that have since changed.

Integrity and access: decide how records are protected from alteration and who can read them, since they may contain sensitive prompts and outputs.

Retention and residency: set retention periods and data residency through your own legal and records policy, and confirm the system can be configured to match.

Producibility: test whether you can retrieve the records for a specific user, system and date range within a reasonable time. A dry run with internal audit shows quickly whether the archive is usable.

How enforcement in the execution path produces evidence

Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants. Each decision to allow, redact, block, rate limit or escalate for human approval applies to an individual interaction or tool call, and enforcement generates the audit-ready evidence as it happens. Policy is enforced at runtime, not reviewed after the fact.

This addresses areas 3, 4 and part of 5 above. It does not replace inventory governance, pre-deployment validation, model lineage or workflow-level tracing.

In production, Trussed has run continuously in a regulated enterprise for more than 18 months at 99.95% uptime on live traffic. It has placed over 100 models under one policy plane and generates more than 30 GB of audit evidence per day. Deployment is self-managed in the customer cloud, with configurable retention and data residency. Evidence maps to HIPAA, NAIC, NYDFS, EU AI Act and NIST frameworks.