What Runtime Enforcement Means for Third-Party AI Models
Third-party AI model governance in insurance covers the controls an insurer applies to AI systems it does not build or fully control, including vendor-supplied underwriting engines, claims triage models, and pricing algorithms. Most current programs rely on static due diligence: vendor questionnaires, model cards, SOC-type reports, and periodic re-certification completed at onboarding and renewed on a fixed schedule, typically annually.
Runtime policy enforcement is a different control category. It evaluates a model's actual inputs and outputs while the model is operating in production, rather than relying solely on documentation describing how the model is intended to behave. Enforcement can intercept, log, or constrain data sent to a vendor model and the decisions or scores returned, independent of whether the vendor has changed anything internally.
The distinction matters because a vendor's underlying model, including any embedded large language model or scoring logic, can be retrained, updated, or reconfigured after an insurer's due diligence cycle is complete. Static attestations describe the model as it existed at review time. They do not confirm that the same model, unchanged, is what continues to run in production.
Where Static Vendor Due Diligence Falls Short
Vendor questionnaires and point-in-time audits remain a reasonable starting control, but they have a structural limitation: they measure the model at a single moment and rely on the vendor's own representations. An insurer typically does not control the vendor's training data, model updates, or internal logic, which makes vendor self-certification an incomplete substitute for the insurer's own operational visibility into how the model behaves once integrated into underwriting, claims, or pricing workflows.
This creates a temporal gap. If a vendor updates a model between annual reviews, and the update changes how the model scores risk or evaluates a claim, the insurer's most recent attestation no longer reflects current production behavior. Static due diligence has no built-in mechanism to detect that change unless the vendor proactively discloses it or the insurer independently monitors production output.
The practical question for risk leaders is not whether vendor attestations are useful, but whether they are sufficient on their own to demonstrate ongoing oversight between review cycles.
NAIC Accountability Expectations Heading Into the Summer 2026 Meeting
NAIC's Model Bulletin on AI Systems and the work of its Big Data and Artificial Intelligence Working Group place accountability for an AI system's outcomes on the insurer using the system, including when the AI is vendor-supplied. That framing shifts the oversight burden toward the insurer rather than treating vendor documentation as a complete discharge of responsibility.
AI governance and vendor oversight have remained active discussion items across recent NAIC meeting cycles, spanning the Innovation, Cybersecurity, and Technology (H) Committee and related working group activity. Risk leaders should treat the upcoming meeting as a checkpoint for confirming how examiners expect insurers to document ongoing oversight of third-party AI, not as the origin of the underlying accountability expectation, which is already established through the Model Bulletin and working group activity.
For compliance purposes, the operative question in an examination is likely to be whether an insurer can distinguish, with evidence, between documentation-based vendor due diligence and evidence of continuous monitoring of actual model behavior in production.
Static Vendor Due Diligence vs. Runtime Policy Enforcement
The two approaches are not mutually exclusive, but they answer different questions. The table below summarizes how each addresses oversight of a third-party model over its lifecycle.
| Capability | Runtime policy enforcement | Static vendor attestation | Key difference |
|---|---|---|---|
| Evaluation timing | Continuous, as the model operates in production | Point-in-time, renewed on a fixed annual cycle | Detects behavior changes between review cycles |
| Dependency | Independent of vendor cooperation or disclosure | Relies on vendor self-certification and voluntary disclosure | Removes reliance on vendor reporting |
| Scope of visibility | Actual inputs and outputs the model processes | Documentation describing intended model design | Confirms behavior, not just stated intent |
| Detecting model changes | Surfaces shifts in scoring or output as they occur | No built-in mechanism between review cycles | Closes the temporal gap between attestations |
| Regulatory evidence | Continuous logs suitable for examination | Onboarding documentation only | Supports evidence of ongoing oversight |