Defining the Evidence Pack
An evidence pack is not a single document; it is a structured set of artifacts that, taken together, show how an AI system was designed, assessed, and operated across its lifecycle. Auditors reviewing a pack are generally looking for two things: that the documentation reflects decisions actually made during design, and that operational records confirm the system behaved as designed once it was running. A pack that only covers one side of that equation is incomplete regardless of how thorough it looks.
Two Evidence Categories: Design-Time and Runtime
Design-time documentation captures the decisions made before a system is deployed: specifications, architecture rationale, intended use statements, and risk assessments. Runtime-generated records capture what happened after deployment: operational logs, policy enforcement outcomes, and monitoring data. Keeping these categories distinct, while tracing both back to the same system or agent identifier, makes it possible to answer an auditor's question about either the plan or the practice without reconciling scattered sources.
| Category | Representative Artifacts |
|---|---|
| Design Documentation | System specifications, architecture decisions, intended use statements |
| Risk Assessment Records | Identified risks, mitigation decisions, review history |
| Change Management Records | Version history for design documents and production releases |
| Runtime Operational Logs | Agent activity, policy enforcement outcomes, monitoring data |
Where Evidence Packs Commonly Fall Short
Most gaps are structural rather than a lack of effort. Design documents are updated but earlier versions are discarded, so change history cannot be reconstructed. Operational logs record system-level events but not agent identity, permissions granted, or tool invocations, which leaves questions about accountability unanswered. Design-time and runtime evidence are stored separately with no shared identifier connecting them. And evidence completeness is only reviewed right before a scheduled assessment, which means gaps are discovered under time pressure rather than corrected in the ordinary course of operations.
Reducing Preparation Burden With Runtime Governance
Continuous runtime logging of agent identity, permissions, and policy enforcement decisions generates operational evidence as systems run, rather than requiring it to be reconstructed after the fact. When this logging is in place from the start, preparation for a conformance assessment becomes a matter of assembling existing records rather than recreating activity history from partial data. This does not remove the need for design-time documentation; it addresses the operational half of the evidence pack that design documents alone cannot cover.
Preparing for a Conformance Assessment
Conformance assessment structures vary by standard and can include self-assessment, second-party review, or third-party certification. Confirming which structure applies to your organization, through accredited certification body guidance rather than assumption, should happen early, since it affects what level of evidence detail is expected. The checklist below reflects the structural practices that keep an evidence pack ready between assessments, not only before one.