Independent assurance over AI controls for boards of directors
Independent assurance on AI controls is credible when a party independent of the teams that build and run AI systems tests the controls against stated criteria, covers both design and operating effectiveness, and samples runtime evidence directly instead of relying on management summaries. Boards should ask what was tested, by whom, against which criteria, over what period, and with what proof.
Separate management assertion from independent testing
Management assertion is a statement by the teams that build and run AI systems about their own controls. Independent assurance is different: a party independent of those teams tests the controls against stated criteria. Boards should be clear about which of their AI control assurances rest on management assertion and which rest on independent testing.
Forms of independent assurance and their limits
Each form of assurance attests to different things, and each has limits. For any form a board is shown, the useful questions are whether it covers design only or also operating effectiveness over a defined period, and whether the assurer sampled runtime evidence directly instead of relying on management summaries. A review of policies alone, or a model review at one point in time, does not show how controls operate in production.
Tie assurance scope to control layers and runtime evidence
Scope should name the control areas under review and the evidence the assurer can expect to examine in each. The table below lists those control areas and the evidence to expect.
| Control area | Evidence to expect |
|---|---|
| Inventory | A list of agents, models, and tools with named owners, reconciled to what is actually running. |
| Agent identity | Distinct identities per agent, with records of credential issuance and revocation. |
| Least-privilege permissions | Granted permissions compared with actual use, plus periodic review records. |
| Runtime policy enforcement | Policy decisions, including blocks and exceptions, recorded at the time of the action. |
| Tool-call governance | Allowed and denied tool calls, with approval records for sensitive tools. |
| Auditability | Logs that are complete, protected from alteration, retained, and available for direct sampling. |
Judge credibility: independence, criteria, scope, and frequency
Credibility depends on who performed the work, what they tested against, what the scope covered, and how current the assurance is. A board should be able to see the following for any assurance it relies on.
| Question | Expected answer |
|---|---|
| Who tested | A party independent of the teams that build and operate the AI systems. |
| What was tested | Controls in production, not only policies or a model review at one point in time. |
| Against what | Named criteria, such as a framework, standard, or internal control set. |
| With what proof | Runtime records the assurer sampled directly. |
Frequency matters as well. Boards should ask how often assurance is renewed and what triggers an interim review after major change.
Questions for management and assurers
- Basis of assurance: Which of our AI control assurances rest on management assertion, and which rest on independent testing?
- Effectiveness type: Does the work cover design only, or also operating effectiveness in production over a defined period?
- Control coverage: Does scope include agent identity, permissions, tool-call restrictions, runtime policy enforcement, and audit logging?
- Independence and criteria: Who performs the work, how is their independence established, and what criteria do they test against?
- Evidence access: Can the assurer sample logs, policy decisions, and exceptions directly?
- Refresh cycle: How often is assurance renewed, and what triggers an interim review after major change?
Where runtime governance fits
Discuss assurance evidence for AI agents
Talk to an expert about how runtime controls and audit logging can produce evidence that independent assurers can sample.
Talk to an Expert