Detecting Underreporting of AI Incidents to the Board

Why a low incident count is ambiguous

A low count in a board report does not show, on its own, whether incidents were rare or whether they went unreported. The gap between what management reports and what independent runtime records show can be measured, but only if the board has both sets of information in front of it.

Showing events detected, reviewed, classified, and escalated lets the board trace a falling count to the stage where it changed.

Red-flag indicators in board reporting

Red-flag patterns tend to appear in how incidents are defined, reviewed, reclassified, and escalated. The questions below help the board test whether reporting is complete and surface these patterns.

Questions to test completeness

• How is an AI incident defined, and which events such as near misses, policy violations, and blocked actions are excluded?
• How many events were reviewed and judged not reportable in the last period, and who made those decisions?
• Can management reconcile the reported count against tool-call logs, policy enforcement decisions, and access anomalies, and explain each difference?
• How often are incidents reclassified or escalated late, and what was the typical delay from detection to board notification?
• Who owns AI incident reporting across security, legal, and engineering, and how are reportability disputes resolved?
• Has internal audit or another independent party tested reporting completeness, and is there a channel that bypasses management?

Reconciling reports against runtime evidence

Reported incidents can be compared against agent tool-call logs, policy enforcement decisions, and access anomalies. The table sets out what each source contributes and how reconciliation and independent assurance fit around them.

Set the taxonomy and thresholds before the data arrives

The board should require a clear incident taxonomy and fixed escalation thresholds, agreed in advance, so that what counts as an incident and when it escalates is not decided after the data arrives.

Practices that give the board verifiable visibility

• Separate the evidence from the reporter: Store tool-call logs and policy decisions where the teams preparing the board report cannot edit them, and keep retention long enough to support audit sampling.
• Reconcile on a fixed cadence: Run the comparison between logs and the incident register every reporting period, and present the unexplained differences to the board alongside the count.
• Report the funnel, not only the total: Show events detected, reviewed, classified, and escalated, so a falling count can be traced to its stage.
• Use independent assurance: Have internal audit or another independent party test completeness periodically by sampling runtime records and tracing them to the register.
• Provide a channel outside management: Give employees and control owners a way to raise AI concerns directly to the audit committee or board, and track those reports separately.
• Review thresholds after every change: Require a board-visible note when definitions, severity criteria, or escalation rules change, with the effect on prior-period counts.