EU AI Act Compliance RFP Questions for AI Vendors: Runtime Evidence and Tool-Call Logs

What the EU AI Act expects buyers to be able to show

Regulation (EU) 2024/1689 sets requirements for high-risk AI systems on risk management, data quality, technical documentation and record-keeping, transparency and information to deployers, human oversight, and robustness, accuracy and cybersecurity. Recital 71 states that comprehensible information on how these systems perform throughout their lifetime is essential for traceability, verifying compliance, monitoring operations and post-market monitoring, and that this requires keeping records and making technical documentation available.

Post-market monitoring adds a further need. The system must actively and systematically collect, document and analyse relevant data on performance throughout the system's lifetime, and that data may come from deployers as well as other sources. Authorities can also act: the Commission's AI Office can send requests for information to verify providers' compliance. A buyer should therefore expect to retrieve evidence quickly, not assemble it from scratch.

On timing, the Commission gives 2 December 2027 as the start date for high-risk obligations, as of November 2026. A Commission proposal of 19 November 2025, the Digital Omnibus on AI, would set fixed deadlines of 2 December 2027 for stand-alone high-risk systems and 2 August 2028 for systems embedded in products. Procurement should align RFP timelines with legal counsel.

The Regulation establishes that records matter. It does not prescribe per-tool-call fields, retention values or data residency terms. The questions below are therefore buyer-defined requirements. Set the specific values with counsel, and state in the RFP whether the vendor acts as provider, deployer or supplier to one, since transparency and monitoring duties refer to both roles.

Runtime evidence versus conventional logs

An application log records what the software was written to emit. It is useful for debugging and operations, but it usually captures requests and errors, not the policy that governed an action or the decision taken. Observability data has the same limit. It describes behaviour after the fact.

Runtime evidence is a record produced at the moment a policy decision is made. It states what was requested, which policy applied, what the enforcement decision was and what the outcome was. The test that separates the two is where the decision happens. If the allow, redact, block or escalate decision is made in the execution path before the action completes, the record is a by-product of enforcement. If decisions are made in a separate review layer, or inferred later from application logs, the record can show what occurred but cannot show that the system was prevented from doing anything else.

One further caution. Test logs and test reports that appear in technical documentation relate to validation and testing. They are not runtime operational records, and a vendor should not offer them as a substitute.

Questions on integrity, retention, residency and deployment

Ask how the vendor makes records tamper-evident and who can alter or delete them, including the vendor's own staff. A substantive answer describes technical controls, such as write protection, access restrictions and a method an auditor can use to confirm that records are complete. A weak answer says data is "secured" without describing how completeness can be checked.

For retention, state the periods you require and ask whether they are configurable and contractually committed. Because the Regulation text does not give values you can copy, legal review should set them. Be wary of fixed retention that the vendor controls, or deletion defaults that cannot be changed.

For residency and deployment, ask where evidence is stored, whether storage location can be chosen, and whether the system runs as SaaS or in your own cloud. Ask for written commitments, not marketing statements. Deployment in your own environment can simplify residency and access questions, but it moves operational responsibility to your team, so ask what that responsibility includes. Finally, ask for independent attestations the vendor holds, and what they cover.

Reading vendor answers: substantive versus weak

Auditor access and mapping to the Regulation are where many responses stay vague. Ask how auditors and regulators retrieve evidence, whether they can do so without vendor intervention, and how fast the vendor can respond to a request. Ask the vendor to map its evidence outputs to record-keeping, transparency, human oversight and post-market monitoring. A mapping that names specific record types is more useful than a general claim of alignment.

• Decision point: Substantive: decision recorded in the execution path, with the policy applied. Weak: decision inferred later from application logs.
• Human oversight: Substantive: escalation and approval recorded as evidence with approver and time. Weak: approvals handled in email or chat outside the record.
• Integrity: Substantive: described controls and a way to verify completeness. Weak: general assurances of security.
• Retention and residency: Substantive: configurable and contractually committed. Weak: fixed defaults or verbal commitments.
• Auditor access: Substantive: defined retrieval process and response time. Weak: access only through support tickets with no timeline.
• Regulatory mapping: Substantive: record types mapped to specific requirement areas. Weak: a statement that the product is compliant.

How Trussed approaches runtime evidence

Trussed enforces policy in the execution path across models, agents, tools and MCP servers. Decisions on individual interactions and tool calls are enforced at runtime, not reviewed after the fact: allow, redact, block, rate limit or escalate for human approval. The audit-ready evidence is generated by that enforcement and is mapped to the EU AI Act, among other frameworks. Trussed supports self-managed deployment in the customer cloud, with configurable retention and data residency.

Trussed has run continuously in a regulated enterprise for more than 18 months and generates more than 30 GB of audit evidence per day. These are useful reference points for scale, but the RFP questions above apply to any vendor, including this one. Ask for sample records and test them against your own fields.