NAIC AI Model Bulletin audit evidence: how runtime policy enforcement produces it

What examiners need to see beyond a written program

The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers was adopted on December 4, 2023. It is a model that state insurance departments issue through their own bulletins. Alaska, for example, issued Bulletin B 24-01 on February 1, 2024. Because adoption is state by state, check the bulletin and operative date that apply to each state where you write business. This position is as of October 2026.

The adopted text expects an AIS Program that addresses governance, risk management controls and internal audit functions. It also lists information and documentation relating to or evidencing the insurer's implementation of and compliance with that program, including documents on monitoring and audit activities. NAIC's AI Systems Evaluation Tool puts the same idea in plain terms: insurers must demonstrate that oversight mechanisms are in place and functioning effectively.

"Functioning effectively" is a statement about operation, not documentation. Policies, model inventories and periodic reviews describe intent. They do not show what an AI system was permitted to do, or what it did, during a particular interaction. Many carriers and brokerages reconstruct that picture afterward from application logs and manual attestations. That approach leaves gaps when an examiner asks for proof of control over third-party models, data use and access.

Mapping evidence records to Bulletin expectations

The table below is a crosswalk to build and test with counsel, not a statement of what the Bulletin prescribes. The Bulletin does not define record schemas. These mappings are a practitioner's reading of how per-interaction records can support the expectations for governance, risk management controls, internal audit, and documentation evidencing implementation and compliance.

• Policy identifier, version and action taken: Shows which control governed an interaction and what it did. Supports evidence that risk management controls operate in practice.
• Identity and tool-call context: Shows who or what acted, on which model or tool, and when. Supports access oversight, including use of third-party models, and monitoring.
• Redaction events: Shows that data-use rules were applied to a specific request. Supports risk management controls over data handling.
• Human approval outcomes: Shows who approved or denied an escalated action. Supports internal audit evidence and clear accountability.
• Policy change history: Shows which version was in force on a given date and who changed it. Supports governance ownership of policy changes and exceptions.
• Exports by date, system, user or policy: Supports producing documentation relating to or evidencing implementation and monitoring and audit activities on request.

Enforcement, logging and after-the-fact review are different evidence

Three sources are often conflated. Application logging records that activity occurred, but it usually does not record which policy governed the activity or what that policy decided. After-the-fact review reconstructs events from whatever was kept, and its completeness depends on what someone thought to retain. Evidence written at the moment of enforcement records the control's own decision, tied to the policy version in force.

The scope matters as much as the distinction. Per-interaction enforcement records cover individual interactions and tool calls. They do not replace the other documentation the adopted text describes, such as model inventories, detailed development documentation, validation and testing of outputs, and evaluation of drift. They also do not establish model lineage or trace an entire agentic workflow from start to finish. Those expectations need their own artifacts, and a crosswalk should say plainly which expectations runtime records cover and which they do not.

That honesty helps in an examination. A narrow, accurate claim that runtime evidence covers monitoring, access control and control operation is easier to defend than a broad one.

Testing readiness and where Trussed fits

The most useful readiness check is a mock examination request. Choose a date, an AI system and a user, then ask the team to produce the policy version in force and the action taken. If the answer requires stitching together application logs and emailed attestations, the evidence is being reconstructed rather than generated. Repeat the exercise for an escalated action and confirm the approver and outcome are present.

Alongside that test, assign owners for policy changes, exception approvals and evidence review. Document who may approve escalated actions and how those approvals are reviewed. Cross-check the program against NAIC's AI Systems Evaluation Tool as one view of what regulators may ask, while recognizing that each state determines how it conducts examinations.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with actions of allow, redact, block, rate limit and escalate for human approval. Its scope is individual interactions and tool calls. Audit-ready evidence is generated by that enforcement and mapped to the NAIC Bulletin among other regimes. It supports self-managed deployment in the customer cloud with configurable retention and data residency. It does not provide transaction-level governance, model lineage or end-to-end agentic workflow tracing, so those remain separate artifacts in your program.