Questions to Ask AI Governance Vendors About the NAIC AI Model Bulletin

Start with what the bulletin asks insurers to show

The NAIC adopted the Model Bulletin on the Use of Artificial Intelligence Systems by Insurers on December 4, 2023. It sets out expectations for a written AIS Program that addresses governance, risk management controls and internal audit functions. It also extends that program to third-party AI systems and data, including the insurer's process for acquiring, using or relying on them.

Adoption happens state by state through individual bulletins, so the dates and text that apply to your carrier or brokerage can differ. Confirm which states govern your business before you write evaluation criteria.

The bulletin does not prescribe an architecture, and responsibility stays with the insurer. A vendor's statement that its product is aligned with the bulletin therefore proves little by itself. The useful test is whether the vendor's output matches what a regulator may request: the written AIS Program, data governance and controls including lineage and quality practices, techniques, measurements and thresholds, and contracts with third-party AI, model or data vendors.

The NAIC's AI Systems Evaluation Tool is a second reference point. It was in a 12-state pilot as of March 2026 and is not shown to be a mandate. Its governance exhibit asks for the governance framework, Board reporting frequency, the policy for and oversight of material AI vendors, and model design and testing. Draft your own answers to those items and check whether a vendor's output would support them.

Governance questions: what does the product evidence, and what is left to you?

Ask the vendor to name the AIS Program elements it produces evidence for: governance, risk controls, internal audit and third-party oversight. For each one, request a sample record, redacted if necessary. Then ask which elements remain manual. A strong answer is specific. It names the artifact, says what generates it, and states plainly where your team still compiles material by hand.

A weak answer describes policy templates, a framework library or a dashboard score. Policy documents describe intent. An examiner testing whether the program operates in practice will want records showing what happened.

Also ask how the product helps you inventory AI assets and identify potentially high-risk models, since the Evaluation Tool asks for that information. Finally, ask whether its reporting can support Board and governance reporting at the frequency your program defines. Confidentiality matters here too. The bulletin protects proprietary and trade secret information, so ask how reports can be shared with governance bodies without exposing vendor internals.

Third-party oversight: coverage and contract terms

The bulletin's third-party provisions reach both AI systems developed by others and third-party data used to develop AI systems. Ask which assets the vendor can cover: models, agents, tools, third-party AI systems and coding assistants. Then ask how it surfaces assets it does not manage. A credible answer describes how gaps are detected. A weak answer assumes the customer supplies a complete inventory.

Check contract terms against the vendor-contract list the bulletin says a regulator may request: representations and warranties, data security and privacy, data sourcing, intellectual property rights, confidentiality and disclosures, and cooperation with regulators. The last item deserves attention. Confidentiality clauses that restrict what you can disclose in an examination can conflict with it. Ask for the vendor's standard position in writing before negotiation.

Also ask how the vendor supports data lineage and quality practices for any data it handles. A vendor's answer does not replace your own oversight. The bulletin addresses your process for relying on third parties, so the question is whether the vendor makes that process easier to run and to prove.

Documentation and examination readiness: evidence, retention, residency and deployment

Evidence is the criterion that separates vendors most clearly. The bulletin does not define record types, retention periods, residency or deployment requirements, so these are buyer criteria. They are reasonable ones for demonstrating that controls operated as designed. Ask what records are created per interaction or tool call. Useful categories include policy decision records, access and tool-call records, human approval records, and the retention and residency settings in force when each record was written.

Then ask who can access the records, how long they are kept, where they are stored and whether the retention period is configurable. Ask whether the records can be exported in a form an examiner can inspect without vendor assistance. A strong answer includes a sample record set and a documented export path. A weak answer points to a dashboard or promises that evidence is available on request.

Deployment model matters because it determines where sensitive prompts, outputs and records reside. Ask whether the product runs as a vendor-hosted service or can be self-managed in your own cloud, and how data residency is set. Ask what happens to evidence if the contract ends. Evidence you cannot retrieve later is not durable.