ISO/IEC 42005 Evidence for NAIC Bulletin Exams: What Standards Artifacts Show and What Runtime Evidence Adds

What an examiner can ask an insurer to produce

The NAIC Model Bulletin on the use of artificial intelligence systems by insurers was adopted on December 4, 2023. It is a model bulletin that individual state insurance departments issue, so the operative text for a carrier is the version its own department has issued. The bulletin sets out the Department's expectations for how insurers govern the development, acquisition and use of AI Systems. It also advises insurers of the information and documentation the Department may request during an investigation or examination.

Section 4 of the bulletin gives guidance on those requests, including in a market conduct action. It also states that a regulator may go beyond the listed items and ask about any specific model, AI System or application, including requests for particular types of information and documentation. That second point matters for evidence design. A policy-level document answers a policy-level question. A question about one model or one application may call for records at that level.

NAIC's AI Risk Evaluation Supplement materials (version 5.0) use a helpful vocabulary. They list a risk assessment item, described as documentation explaining the company's risk assessment process for its use of AI Systems. They also list two evidence categories: control design evidence and control operation evidence. This guide uses those two terms throughout. The materials do not say how examiners weigh one category against the other, so treat the split as a way to organize your own response, not as a statement of examiner preference.

What ISO/IEC 5338 and ISO/IEC 42005 contribute

ISO/IEC 5338 addresses AI system lifecycle processes. ISO/IEC 42005:2025 addresses AI system impact assessment. Both are voluntary standards. They are useful because they give a carrier a structured, recognizable way to define how AI systems are built, changed and assessed. They are not examination checklists, and nothing about conformance to either one tells an examiner that the Bulletin or SR 26-2 has been satisfied.

From a lifecycle process standard, you can expect artifacts that describe defined processes: who does what at each stage, which procedures apply and which outputs are produced. From an impact assessment, you can expect documentation of assessed impacts and intended mitigations. Both are control design evidence. They show that the organization thought about the system, defined its controls and recorded its reasoning.

By their nature, neither artifact is a record of production behavior. An impact assessment completed before deployment says what the controls are meant to do. It cannot say what happened last Tuesday when a particular request reached a model or a tool. That is the gap between written governance and demonstrable runtime behavior, and standards documentation alone does not close it.

Mapping evidence needs to artifact types

The table below is a starting structure, not a clause-level mapping. Before you rely on any row, confirm it against the text of each standard and the Bulletin version your department has issued.

• Defined processes for developing, overseeing and validating AI: Lifecycle procedures can support this. They show the process exists, not that it was followed for a given system.
• Risk assessment process documentation: Impact assessments can support this. They show assessed impacts and intended mitigations, not production outcomes.
• Thresholds, measurements and controls the insurer actually uses: Process documents can state them. Records that the controls were applied are a separate artifact type.
• Questions about a specific model, AI System or application: Policy-level documents give context. Interaction-level records can answer more directly.
• Human oversight: Procedures define who reviews what. Approval records show who approved, when and on what basis.
• Control operation in production: Standards artifacts fall short on their own. This evidence must be generated during production operation.

Where runtime enforcement evidence fits

Enforcing policy in the execution path produces a record as a side effect of the control itself. When each interaction or tool call is evaluated against policy and the outcome is allow, redact, block or escalate for human approval, that decision can be recorded at the moment it is made. The record is not a reconstruction after the fact. It is the control's own output, which is why it sits naturally in the control operation category.

This is an analytical position, not a stated regulator preference. The materials reviewed here do not say how examiners treat runtime evidence relative to process documentation, and legal and compliance should review any claim on that point before it appears in an examination response. The practical argument is narrower. If a regulator asks about a specific application, interaction-level records let you show the policy applied and the outcome, where a policy document can only describe the intent.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers, at the level of individual interactions and tool calls. That enforcement generates continuous audit-ready evidence mapped to NAIC, among other regimes. Its scope is enforcement and the evidence it produces. It does not replace the lifecycle procedures or impact assessments described above, which remain your design evidence.

Building the evidence-mapping matrix

Start with the examination request categories as rows. Use four columns: evidence from ISO/IEC 5338 lifecycle outputs, evidence from ISO/IEC 42005 impact assessments, runtime evidence, and gap. The same structure extends to SR 26-2 by adding its request categories as further rows once you have read its text. Populate cells only after reading the standards and the adopted regulatory text, and treat community crosswalks as orientation aids, not authoritative mappings.

• Tag every artifact as control design evidence or control operation evidence, and flag any control that has only design evidence.
• Work from the Bulletin text your state department has issued, not from draft request lists.
• Assign an owner for keeping the written AIS Program consistent with what runtime records show.
• Include human approval escalations in oversight records, with approver, time and basis.
• Test the export path: pull the policy, outcome and approval for one named interaction, end to end.
• Confirm retention periods and export formats with counsel, since they depend on your obligations.

Tradeoffs and limits

Separating the two evidence types makes a response easier to defend, but it adds work. Runtime evidence has to be retained, indexed and exported, and the volume can be large. Decide early which records must be produced on request and how long they are kept, rather than discovering the answer during an examination.

The matrix also exposes uncomfortable gaps. A control with a thorough impact assessment and no operation records is a finding waiting to happen, and a mismatch between the written program and observed behavior is harder to explain than a missing document. Finding these gaps internally, before an examiner does, is the main value of the exercise.

Finally, keep claims proportionate. The standards give structure, the Bulletin and SR 26-2 set the expectations, and runtime evidence shows operation. No single artifact does all three jobs.

Common questions

Short answers to questions compliance teams raise when scoping an evidence response.

• {"question":"Does conformance to ISO/IEC 42005 satisfy the NAIC AI Model Bulletin?","answer":"No. It is a voluntary standard, not a regulatory requirement. An impact assessment can support the risk assessment documentation an insurer maintains, but it records assessed impacts and intended mitigations, not production behavior. Treat it as design evidence alongside other artifacts."}
• {"question":"What is the difference between control design and control operation evidence?","answer":"Design evidence shows what controls are defined and intended, such as procedures and assessments. Operation evidence shows the controls were applied, such as records of enforcement decisions on specific interactions. NAIC's supplement materials list both as evidence categories."}