How to Score an EU AI Act Vendor Demo: Proof Tests for Article 12 Logging and Article 14 Oversight

Why a demo needs to be tested, not watched

Vendor demos for high-risk use cases tend to show dashboards and slides. A dashboard shows that data can be displayed. It does not show that the system produced the underlying records on its own, that those records will survive, or that a person could have intervened while the system was running.

The text of Regulation (EU) 2024/1689 gives you two anchors for demo requests. High-risk AI systems must technically allow for the automatic recording of events (logs) over the lifetime of the system. They must also be designed and developed, including with appropriate human-machine interface tools, so that natural persons can effectively oversee them while they are in use. The brief for this guide maps these to Article 12 and Article 14.

The wording matters for how you test. "Automatic" means records should be generated by the system, not assembled by a vendor engineer for the meeting. "In use" and "human-machine interface tools" point to oversight that exists in the product while it operates, not only in a policy document or a periodic review.

The tests below go further than the legal wording. They are editorial test design meant to show whether records can serve as audit evidence and whether oversight works in practice. As of November 2026, work from the consolidated text of the Regulation dated 27 July 2026 and record that version in your assessment file. Ask which version the vendor's readiness claim relies on.

A weighted EU AI Act vendor scorecard

Score each category from 0 to 3. A score of 0 means claimed only. A 1 means shown in slides or a static screenshot. A 2 means demonstrated live in part. A 3 means demonstrated live with an artifact you can keep. The weights below are a starting point for a high-risk AI system vendor assessment, and you should adjust them to your use case and risk appetite.

Apply a gate before totalling points. If a vendor scores 0 or 1 on event coverage, intervention or oversight evidence, remove it from the shortlist regardless of its total. A high score elsewhere should not offset the absence of the two capabilities the Regulation's text centres on.

• Article 12 event records (30%): Coverage, traceability, tamper resistance, retention and exportability, tested live.
• Article 14 oversight (30%): Intervention, stop or override, escalation and oversight evidence, tested through the interface.
• Documentation artifacts (20%): Technical documentation and conformity documentation provided before shortlisting and kept up to date.
• Deployment, retention and residency options (10%): Where the system runs, where records are held and who controls retention settings.
• Role clarity and contract terms (5%): Written allocation of who holds records and who produces them on request.
• Version discipline (5%): Readiness claims tied to the consolidated text and its amendments.

Evidence to request before shortlisting

A live demo shows behaviour. Documents show what the vendor will stand behind. Providers of high-risk AI systems must draw up technical documentation before placing the system on the market or putting it into service, and keep it up to date. They may use it to demonstrate compliance through conformity assessment or market surveillance activities. On a reasoned request from a competent authority, they must provide the information and documentation needed to demonstrate conformity, in an official Union language the authority can easily understand.

That gives you a concrete request list: the technical documentation, the conformity documentation, a sample export of records from your demo session, and the oversight record from your intervention test. Ask in which languages and formats the documentation could be supplied to an authority. Then put the obligation in the contract, so the vendor commits to producing this evidence when you or a regulator need it.

Be cautious about role allocation. Which party holds the records and which party produces them depends on whether the vendor is the provider and you are the deployer for the use case. Settle this in writing during procurement, with legal counsel, rather than leaving it to assumptions made in a demo.

Finally, separate evidence that exists from evidence that is audit-ready. Audit-ready means durable, attributable to a specific actor and event, and retrievable years later by someone who was not in the room. Score what you were given, not what you were told is available.

Where runtime enforcement fits in the evaluation

The two test sets point to the same design question: does the product enforce policy while the system acts, and does that enforcement itself produce the evidence? When a control sits in the execution path, the allow, block or approval decision and its record come from the same event. When controls are reviewed afterwards, the record and the decision can drift apart.

Trussed approaches this by enforcing policy in the execution path for individual interactions and tool calls: allow, redact, block, or escalate for human approval. Audit-ready evidence is generated by that enforcement and mapped to the EU AI Act, among other frameworks. Retention and data residency are configurable, and deployment is self-managed in the customer's cloud. You can apply the same proof tests to any vendor, including this one. Ask for the live escalation, the held action and the exported record.

Treat any approach as one layer of an assessment. Runtime enforcement of individual interactions does not replace the technical documentation, conformity documentation and contractual commitments described above.