Buyer Evaluation Criteria for ISO/IEC 23894 Risk Management Tooling

Treat 23894 as a process, not a product checklist

ISO/IEC 23894:2023 provides guidance on how organizations that develop, produce, deploy or use AI products, systems and services can manage risk specific to AI. It also aims to help organizations integrate risk management into their AI-related activities and functions. As of November 2026, it remains the published edition in the ISO catalogue.

Because it is guidance, no product can be 23894 compliant in a certifiable sense. A tool can support one or more activities in the risk management process, and a vendor claim should be read that way. ISO/IEC 42001 is a different kind of document. It specifies requirements for establishing, implementing, maintaining and continually improving an AI management system. Where an organization uses both, the management system supplies policies, objectives and processes, and 23894 informs how risk is handled inside them. Buyers should require capabilities that serve the risk process itself and treat management-system features as a separate decision.

One sequencing point matters for procurement. Define risk tolerances, response plans, roles and review frequency before selecting a tool. Tooling that is configured to undefined criteria produces records but not decisions.

Mapping process activities to tooling capabilities

The table maps each activity to the capability a buyer should look for. Capabilities are described by function rather than product label, because vendors in GRC, governance, testing and runtime control use overlapping language.

• Context: An inventory of AI systems, owners and uses, plus configurable risk criteria. Scales should follow organization-defined criteria, including qualitative ones such as red-amber-green.
• Risk assessment: Structured assessment workflows and ingestion of testing and evaluation results, covering both pre-deployment and ongoing evaluation. These outputs inform treatment but are not treatment.
• Risk treatment: A record of the chosen response (mitigate, transfer, avoid or accept) and, separately, a mechanism that acts on live AI activity.
• Monitoring and review: Tracking of existing, unanticipated and emergent risks based on actual performance in deployed contexts, with periodic review and defined roles.
• Recording and reporting: An organized, secure repository for risk and system documentation, including residual and accepted risks, that can be retrieved for review.

Documented treatment versus enforced treatment

The most useful distinction for a buyer is between a tool that records a treatment and a tool that applies it. A workflow can capture that a risk will be mitigated by restricting a data type or requiring human approval for a class of action. That record does not stop anything. Enforcement happens only when a control operates on the live interaction, such as a model call or a tool call, and can allow, redact or block it.

Assessment and testing sit on a third axis. They produce findings that feed treatment decisions, and they matter before deployment and on an ongoing basis, but they do not themselves change what a production system does. Ask of every product which of three things it does: document treatments, test them, or act on live activity.

A single tool rarely spans all three. Workflow-oriented products are the natural home for context, assessment records, residual risk approval and reporting. Testing products address evaluation before and after release. Runtime controls address treatment and part of monitoring in production. Plan for a combination and decide where the system of record sits, rather than expecting one purchase to cover the whole process.

Evidence is an output of control

Evidence for AI risk treatment is most credible when it is produced by the control itself, not assembled afterward for a report. If a treatment is enforced on each interaction, the record of that enforcement is the proof that the treatment was applied. If a treatment is only documented, reviewers must infer from attestations that it operated.

Test four properties in any tool. Completeness: does every governed interaction generate a record, including blocked and escalated ones? Durability: can records be altered or lost, and how is the repository secured and organized? Retention: is the period configurable to your regulatory obligations? Data residency: can you choose where records are held? Then ask whether an auditor can examine the records without the vendor's involvement.

Trussed is one example of runtime policy enforcement in the execution path. It enforces policy across models, agents, tools, MCP servers, AI sites and coding assistants (allow, redact, block, rate limit, or escalate for human approval) and generates audit-ready evidence from that enforcement, mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. It deploys self-managed in the customer cloud with configurable retention and data residency, and has generated 30+ GB of audit evidence per day in production.

It does not cover pre-deployment testing or red-teaming, and it is not a governance workflow or GRC system. It does not replace the system of record for context, assessment and residual risk approval. It addresses treatment and the evidence of treatment on live activity.

Questions to put to vendors

Use the same questions with every vendor and require answers in writing. Vague answers to the first two usually indicate that the product documents rather than enforces.

• Which process activities does the product support, and what does it explicitly not cover?
• How does it show that a documented treatment was applied to live AI activity and then monitored in deployed use?
• How are residual and accepted risks represented, and who approves and reviews them?
• Which mappings to ISO/IEC 23894, ISO/IEC 42001, NIST AI RMF and the EU AI Act are provided, who wrote them, and how are they maintained? Treat mappings as inputs to legal review, not substitutes for it.
• Where are evidence records stored, for how long, and can an auditor examine them independently?
• What happens to an interaction if the control is unavailable or cannot reach a decision?