Assessing Third-Party and Vendor AI Systems Under ISO/IEC 42005 and ISO/IEC 23894

How the two standards apply to systems you do not build

ISO/IEC 23894:2023 gives guidance on how organizations that develop, produce, deploy or use AI products, systems and services can manage AI-specific risk. It also describes processes for integrating risk management into AI-related activities and functions. ISO/IEC 42005:2025 is presented by ISO as a methodology for identifying, analyzing and documenting the effects of individual AI systems, and it is designed to work together with ISO/IEC 42001. As of November 2026, both appear in the ISO/IEC JTC 1/SC 42 catalogue.

Neither standard is described as a vendor-assessment procedure. That gap is where risk teams have to make decisions. A workable division is to let 23894 supply the process (risk criteria, assessment, treatment, monitoring) and let 42005 supply the structure for documenting the impact of one specific system in one specific use. Treat both as guidance, not as a checklist a vendor can satisfy on your behalf.

The NIST AI RMF supports treating vendor AI as in scope. It states that parties should manage risk in AI systems they develop, deploy or use, whether standalone or integrated components. That covers embedded features and APIs, not only models you select deliberately. NIST materials also add third-party specifics that the ISO guidance leaves open, and the steps below draw on them where useful.

Separating what attestations can resolve from what they cannot

A vendor questionnaire captures a point in time. It can tell you how the vendor says the system was developed, tested and secured, and what the vendor commits to do when problems arise. It cannot show what the system was permitted to do in your environment or what it actually did there.

For each risk in your assessment, assign one of three treatment paths. The first is risks the vendor controls and can reasonably attest to, such as internal testing practices, documentation and notification commitments. The second is risks addressed by contract and process, such as incident response ownership, audit rights and change notice. The third is risks that depend on how your people and systems use the vendor AI, and therefore need controls and evidence inside your own environment.

Examples of the third path include sensitive data sent to a vendor API, an assistant invoking a tool with broad permissions, and usage patterns that drift from the intended use you documented. A vendor attestation cannot close these, because the vendor does not see your policy. This sorting step is the most useful output of the assessment for shared responsibility, because it makes the boundary explicit and assigns an owner on each side.

Treatment decisions, residual risk and reassessment

NIST describes risk response options as mitigating, transferring, avoiding or accepting. Record the chosen option and the reasoning for each risk. Transferring risk through a contract changes who bears cost, but the risk still belongs in your record. NIST MANAGE 1.4 asks that negative residual risks, meaning the sum of all unmitigated risks, to downstream acquirers and end users be documented. Procurement guidance adds that a risk owner with decision-making power should recognize risks and mitigations, including the go/no-go decision, and that tenders should ask suppliers to identify risks and explain mitigations.

NIST Playbook actions also include monitoring third-party AI for negative impacts and decommissioning third-party systems that exceed risk tolerances. Build the exit condition into the treatment record in advance, so that a tolerance breach has a defined consequence.

On reassessment, define triggers yourself and document them. Reasonable candidates are a vendor model or version change, a new embedded AI feature, a wider use case, a new data category, and a security or incident event. NIST guidance supports continuous monitoring and periodic review with clearly defined roles and frequency. More triggers improve coverage but increase workload, so tie them to the risks you rated highest.

Where runtime policy enforcement supports the assessment

Trussed enforces policy in the execution path on interactions and tool calls involving models, agents, tools, MCP servers, AI sites and coding assistants. Policy is enforced at runtime, not reviewed after the fact. Actions are allow, redact, block, rate limit, or escalate for human approval. This maps to the third treatment path above: risks that depend on your own use of vendor AI.

Enforcement also produces continuous, audit-ready evidence of what was allowed and what happened, mapped to frameworks including NIST, HIPAA, NAIC, NYDFS and the EU AI Act. For a regulated enterprise, that evidence can support the monitoring record behind step 6, show how a treatment decision was applied in practice, and indicate when observed use departs from documented intended use. It runs self-managed in the customer cloud, with configurable retention and data residency. Trussed has operated 18+ months in continuous production in a regulated enterprise, with 100+ models under one policy plane.

The boundary matters. Enforcement covers individual interactions and tool calls. It does not replace the impact assessment, vendor documentation, or contractual commitments, and it does not establish model provenance or lineage.