AI Reproducibility vs Reconstructibility

Where reproducibility holds and where it breaks

Reproducibility is a familiar standard from conventional software and from research. Given the same code, configuration and input, a deterministic system returns the same output, so an investigator can re-run a past event and compare.

LLM and agent systems weaken each condition. Sampling introduces randomness unless it is tightly constrained, and even constrained inference may not be bit-identical across hardware or serving changes. The model behind an endpoint can be updated or replaced. Prompts and system instructions are edited. Retrieved context changes as documents, indexes and databases change. Tool calls return different results depending on the state of the systems they touch. Conversation history is itself an input, and it may be incomplete or influenced by an attacker.

The result is that a regulated enterprise often cannot re-run an interaction and obtain the original output, even with good engineering. For agents, re-execution has a further problem. A tool call that changed a record or triggered a transaction should not be repeated in order to investigate it. Reproducibility remains useful for development and testing. It is a weak foundation for audit evidence about something that already happened in production.

Reconstructibility as the achievable evidentiary standard

Reconstructibility does not ask the system to behave identically a second time. It asks whether the enterprise can show, from records made at the time, what happened in a specific interaction or tool call. That means what went in, what context the system relied on, which policies applied and what decision they produced, and what came out or what action was taken.

The distinction matters for requirements. The EU AI Act's language for high-risk AI systems centres on technically allowing automatic recording of events (logs) over the lifetime of the system. Its recitals tie records and technical documentation to traceability, compliance verification and post-market monitoring. This is the vocabulary of records and traceability. As of October 2026, a consolidated text dated 27 July 2026 is the current reference, and obligations should be confirmed against it with legal counsel.

In practice, an auditor examining one interaction needs a trustworthy account of what occurred. A requirement phrased as "prove we can get the same answer again" is one that non-deterministic systems cannot meet, and it tends to push teams toward architectures that capture the wrong records.

Reproducibility and reconstructibility compared

The two standards differ in what they set out to establish, what they depend on and how they fail. For non-deterministic AI, only one of them can be satisfied consistently.

• Purpose: Reproducibility: obtain the same output by re-running. Reconstructibility: establish what happened, with what inputs, context and policy.
• Required artifacts: Reproducibility: pinned model version, fixed prompts, frozen context, controlled sampling, stable tool results. Reconstructibility: runtime records of inputs, context, policy decision, policy version, tool call parameters, outputs, identity and time.
• Failure modes: Reproducibility: fails when the model, retrieved data, tool state or sampling changes, and is unsafe for irreversible tool calls. Reconstructibility: fails when records were not captured at the time, are incomplete, or can be altered.
• Audit usefulness: Reproducibility: limited, because a matching re-run does not show what happened originally. Reconstructibility: high, because it is evidence about the actual event.

Evaluating whether runtime evidence is sufficient

Capturing the fields above is necessary but not enough. Evidence has to survive examination, and a short set of tests helps when reviewing any runtime control approach.

First, is capture automatic and continuous, or does it depend on optional settings and after-the-fact collection? Second, does the record show the context the system relied on, rather than only the final output? Third, does it show what a tool call was permitted to do as well as what it did, which matters most for actions that cannot be undone? Fourth, is each record linked to a versioned policy and to current technical documentation, so an auditor can see both the design and the behaviour?

Fifth, is the evidence durable and tamper-resistant? Reasonable questions include who can modify or delete records, how retention and data residency are configured, whether integrity can be checked independently, and where the records are stored. No universal standard defines these criteria, so ask for the supplier's own evidence and test it against your examination requirements.

There are tradeoffs. Full context capture increases storage and raises data protection questions, because records may contain sensitive content. Decide on retention, access and redaction rules before deployment.

Where runtime policy enforcement fits

Trussed enforces policy in the execution path for individual AI interactions and tool calls, across models, agents, tools, MCP servers, AI sites and coding assistants. Each interaction is evaluated at runtime and the outcome is allow, redact, block, rate limit or escalate for human approval. The evidence is produced by that enforcement, so it records what the interaction or tool call was allowed to do and what it did, rather than being assembled after the fact.

The audit-ready evidence is mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. Deployment is self-managed in the customer cloud, with configurable retention and data residency. This scope is per interaction and per tool call. It does not cover model lineage, dataset provenance or end-to-end tracing of agentic workflows, which are separate questions from the one addressed here.