Linking Pre-Deployment Validation Results to Production Runtime Evidence for the Same Agent

Why validation and runtime evidence end up disconnected

Validation, testing and red-team exercises usually run in one set of systems. Production enforcement and activity records live in another. Each can be sound on its own, yet an auditor asking whether the agent in production is the agent that was tested has nothing that joins the two. The gap is one of identity and reference, not volume of evidence.

The expectations behind this are consistent. NIST AI RMF 1.0 says AI systems should be tested before deployment and regularly while in operation, with measurements demonstrated under conditions similar to the deployment setting. NIST AI 600-1 recommends a document retention policy that keeps TEVV history, and suggests sharing pre-deployment test results with those who hold release approval authority. The EU AI Act text requires testing of high-risk AI systems against prior defined metrics and probabilistic thresholds, and its recital 71 ties records and technical documentation to traceability and post-market monitoring. NIST AI 800-4 notes that pre-deployment evaluations mostly run in controlled environments and points to feedback loops between pre-deployment and post-deployment results.

None of these prescribes a field list. The linkage model below is a design proposal that satisfies the principle: a validation result is only useful as evidence if it can be tied to a specific release, and runtime records are only useful if they can be tied back to it.

Define the identifiers that make up an agent release

Treat the agent as a release made of components, not as a name. Anything that can change behavior should be part of the release identity, and a change to any of them should produce a new identity or trigger a defined revalidation decision. Record these in a release manifest that is created at build or promotion time and never edited afterward.

• Agent release version: An immutable identifier assigned at promotion. It is the primary key that validation and runtime records both reference.
• Configuration hash: A digest of the effective configuration, including system prompts and parameters, so a changed setting is visible as a changed value.
• Model identifier and version: The exact model the agent calls, recorded as pinned by the deployment rather than as a family name.
• Tool and MCP server inventory: The tools and MCP servers the agent may call, with their versions or definitions, as validated.
• Policy set version: The version of the policy set assumed during validation, which is the set that must be enforced at runtime.
• Validation reference: A pointer to the test plan, metrics, thresholds, results and approval record for this release.

Detect drift between the validated and deployed agent

Drift is any difference between what was validated and what is running. Common sources are changed configuration or prompts, a model version that changes underneath the agent, and added or modified tools and MCP servers. A policy set that differs from the one assumed in validation is the same kind of problem. Separately, even an identical release can behave differently in production than in a controlled test, which is why comparison with post-deployment monitoring matters.

Three controls cover most of it. At deploy time, compare the configuration hash, model, tool inventory and policy set version against the manifest, and block or flag a mismatch. At runtime, record the release reference on every event so a record that does not match a validated release is itself visible. On a schedule, compare validation results with production monitoring results as a recurring feedback loop, and define which kinds of change require revalidation before the release identifier is reissued.

Be explicit about what is self-asserted. A log written by the agent about its own configuration is a component's assertion. A record produced by an independent enforcement point, or a deployment check against the manifest, is stronger. Reviewers should be able to tell the two apart.

Where runtime policy enforcement fits, and what sits outside it

Trussed enforces policy in the execution path across models, agents, tools and MCP servers: allow, redact, block, rate limit, or escalate for human approval, at the level of individual interactions and tool calls. That enforcement generates audit-ready evidence, mapped to frameworks including NIST and the EU AI Act. Because policy is enforced at runtime, not reviewed after the fact, each decision record shows which policy was applied and what was allowed or done. With the release identifier and policy set version available to correlate, those records can be referenced against a validated release. This is the runtime half of the linkage, and it addresses the question of whether the policies assumed in validation were the ones actually enforced.

The boundaries matter. Trussed is not a pre-deployment testing product, so validation design, test execution and red-teaming stay with your existing tools. It does not provide model lineage or dataset provenance, and it does not provide end-to-end tracing of agentic workflows. The release manifest, the approval record and the deployment drift check remain your design responsibility. Enforcement evidence is one input to the chain, not the whole chain.

On timing, EU AI Act high-risk dates are subject to proposed amendment. As of November 2026, the Commission's Digital Omnibus on AI proposes 2 December 2027 for stand-alone high-risk AI systems and 2 August 2028 for those embedded in products. The Council agreed its position on 13 March 2026, and adoption rests with the legislators. Build the linkage to the testing and record-keeping expectations regardless of how the dates settle.