Running an EU AI Act serious incident tabletop exercise for insurers

What the exercise is actually testing

Most incident drills check whether a runbook exists. This one should check whether people can make three decisions under time pressure and back them with records: is this a serious incident, who is obliged to report it and when, and what did the AI system do during the incident window.

The regulatory anchor is Article 73 of Regulation (EU) 2024/1689. Providers of high-risk AI systems must report serious incidents to the market surveillance authorities of the Member States where the incident occurred. Deployers that identify a serious incident must immediately inform the provider first, then the importer or distributor and the relevant authorities. Both duties sit in one event, so an insurer that is a deployer of a vendor's underwriting model still has obligations of its own.

The relevant systems are those the Act's Annex III treats as high-risk, which includes AI used for risk assessment and pricing in life and health insurance. As of November 2026, the Digital Omnibus on AI proposes application dates of 2 December 2027 for stand-alone high-risk systems and 2 August 2028 for those embedded in products, and a provisional agreement between Council and Parliament was announced on 7 May 2026. Confirm the dates that apply to you before fixing a calendar, but the exercise is worth running ahead of them because the gaps it exposes take time to close.

Scope the scenario and assign roles

Pick a system the insurer really operates, such as a model that scores applicants for life or health cover or one that feeds pricing. Generic scenarios produce generic answers. Write down the intended purpose, the instructions for use, the human oversight arrangements and the data flows to any vendor.

Then assign roles explicitly. For each system, record whether the insurer is the provider, the deployer or both, and name the individuals who may declare a serious incident, escalate it and sign off a report to a market surveillance authority. Deployers must use high-risk systems in accordance with the accompanying instructions for use, so include the person who can say whether usage matched them.

If a third-party general-purpose model sits behind the system, add it to the role map. Providers of general-purpose models with systemic risk carry separate incident tracking and reporting duties to the Commission and national authorities, and participants should be able to say which party holds which duty.

Design injects and the reporting-clock decision

Use at least two injects. The first should be built to sit clearly inside the Act's serious incident definition in Article 3, for example a pricing or risk-assessment system producing outputs that lead to harmful decisions about a group of policyholders, surfaced by a complaint and later by a vendor notice. The second should be a near-miss, such as anomalous outputs caught by an underwriter before any decision reached a customer. Have counsel confirm where each inject falls against the definition before the session, and score the team's classification rationale as well as the answer it reaches.

Add a third inject involving real-world testing of a high-risk system, since serious incidents found during such testing must also be reported under Article 73.

The reporting-clock decision point needs three recorded times: when each party became aware, when the causal link or its reasonable likelihood was assessed, and when a report would be filed. A provider's report is due immediately after that causal link or reasonable likelihood is established, and in any event no later than 15 days after the provider or, where applicable, the deployer becomes aware. Test how the team judges reasonable likelihood on incomplete information. Vary severity across injects too, because the reporting period must take account of it, and check whether the team shortens its timeline accordingly.

Score evidence gaps, not just decisions

The central question for the evidence round is what the AI system was permitted to do and what it did. Ask participants to produce the system configuration in force, the applicable instructions for use, and interaction-level behaviour for the incident period. Also ask for proof that human oversight was exercised by staff who were equipped and enabled to do it, which the Commission's AI Act FAQ identifies as a requirement.

Time each retrieval and note who had to be asked. Because the deployer must inform the provider first, map in advance who can retrieve interaction records, from which system, and how quickly, including records held by the vendor. Score each item as available, partial or missing, and record every gap as a finding with an owner.

Insurers operating across jurisdictions can run the same scenario against their obligations under the NAIC AI Model Bulletin, NYDFS Circular Letter 7 and Colorado SB 24-205. The evidence questions above are not specific to the EU, so one exercise can feed several compliance files, but map each regime's own requirements separately rather than assuming they match.

Where runtime enforcement and audit-ready evidence fit

If the evidence round shows that records of what the AI did are scattered, incomplete or only reconstructable after the fact, the gap is architectural rather than procedural. Controls that act in the execution path can narrow it, because the same step that applies policy also produces the record.

Trussed enforces policy in the execution path for individual interactions and tool calls: allow, redact, block, rate limit, or escalate for human approval. That enforcement generates audit-ready evidence mapped to the EU AI Act, NAIC and NYDFS frameworks, and it can be deployed self-managed in the customer's cloud with configurable retention and data residency. In an exercise, the practical test is simple: could the team answer the 'what was it allowed to do and what did it do' questions from those records, within the time the scenario allows?

Treat that as a question the exercise answers, not an assumption. Re-run the scenario after remediation and compare retrieval times and gap scores with the first run.