How Do We Know Our AI Has Not Changed Since the Board Approved It?
You know only if you can compare production to a recorded, versioned baseline of what was approved, and that comparison runs continuously. This requires a baseline that covers the model, instructions, tools, permissions, retrieval sources, and policies, change controls before deployment, runtime checks and enforcement after it, and tamper-evident logs tied to a baseline version.
Board approval is a baseline, not a state
Board approval records what was approved at a point in time. Knowing that the AI has not changed since then depends on comparing production to a recorded, versioned baseline of what was approved, and on running that comparison continuously rather than once.
In practice this takes four connected parts: a baseline, change controls before deployment, runtime verification and enforcement after deployment, and tamper-evident evidence tied to a baseline version.
Define the approved baseline
The baseline is a versioned record of every component the board approved. It should be specific enough to retrieve the exact configuration that was approved. It covers:
- Model: the model that was approved.
- Instructions: the prompts that shape its behavior.
- Tools: the tools the AI can call, including MCP servers.
- Permissions: what the AI and its agents are allowed to do.
- Retrieval sources: the sources the AI can draw on.
- Policies: the policies that apply to its operation.
Because the baseline is versioned, checks and logs can reference a specific baseline version.
Map each changeable component to detection and control
Every component in the baseline can change. For each one, you need a way to detect in production when it differs from the baseline, and a control that determines whether the change is allowed. The four stages below connect approval to attestation.
| Stage | Description |
|---|---|
| Baseline | A versioned record of every component the board approved. |
| Change control | Edits are reviewed and mapped to a baseline version before release. |
| Runtime verification | Production state is compared with the baseline and enforced. |
| Evidence | Tamper-evident logs show conformance over time. |
Separate change management from runtime verification
Change management and runtime verification answer different questions, and both are needed.
Before deployment
Change controls review edits and map them to a baseline version before release.
After deployment
Runtime checks compare production state with the baseline, and enforcement acts on differences.
Keep evidence that can withstand review
Audit logs should be tamper-evident and tied to a specific baseline version, so they show conformance over time. This is what allows you to give a board, audit committee, or regulator evidence of continued conformance without a manual reconstruction effort.
Test whether your current controls produce evidence
Use these questions to check your current controls.
- Can you retrieve a versioned record of the exact model, prompt, tools, MCP servers, permissions, retrieval sources, and policies that were approved?
- Can you detect, in production, when any one of those components differs from the baseline?
- Are unapproved changes blocked at runtime, or only reported after the fact?
- Are audit logs tamper-evident and tied to a specific baseline version?
- Could you give a board, audit committee, or regulator evidence of continued conformance today, without a manual reconstruction effort?
Assess your runtime enforcement and audit trail
If your answers to the checklist show gaps in agent permissions, tool governance, or auditability, a conversation about runtime governance can help scope next steps.
Talk to an Expert