Implementation Guide

    ISO/IEC 5338 Incident and Problem Management for AI Systems

    ISO/IEC 5338 defines AI system life cycle processes but does not prescribe an incident or problem workflow. Security teams map its operation, monitoring and maintenance-related processes to six steps: detect and classify, contain, investigate, find root cause, correct, and close. Each step needs evidence about individual interactions and tool calls. Policy enforced in the execution path produces that evidence as a by-product of every decision.

    See How Evidence Is Produced Talk to an Expert
    1. {"title":"1. Detect and classify","description":"Open an incident from a policy violation, an anomalous tool call, a user report or a monitoring alert. Assign a category and severity using a scheme your organisation defines and documents."}
    2. {"title":"2. Contain","description":"Limit further harm: block the offending action class, restrict a tool or agent, or route sensitive actions to human approval. Record who decided and when."}
    3. {"title":"3. Investigate","description":"Reconstruct the specific interaction or tool call: inputs, outputs, the policy that applied, the decision taken and any human involvement."}
    4. {"title":"4. Establish root cause","description":"Determine whether the cause was a policy gap, a tool permission, a prompt pattern, a model behaviour change or an upstream data issue. Open a problem record if the cause is shared."}
    5. {"title":"5. Apply corrective action","description":"Change policy, permissions, configuration or the system itself. Assign an owner and a due date, and define how you will verify the fix."}
    6. {"title":"6. Close and review","description":"Confirm the evidence supports closure, retain the incident file and feed lessons into monitoring and policy design."}
    7. Interaction record: timestamp, calling user or agent identity, target model or tool, and the request and response involved.
    8. Policy decision: which policy applied, its version, and whether the outcome was allow, redact, block or escalate.
    9. Tool-call record: the tool invoked, the parameters passed and the result returned.
    10. Redaction and block detail: what was removed or stopped, and why.
    11. Human approval record: who approved or rejected an escalated action, when, and with what context.
    12. Incident linkage: references tying evidence to the incident, problem and corrective action records.

    What ISO/IEC 5338 gives you, and what it leaves to you

    ISO/IEC 5338:2023, Information technology, Artificial intelligence, AI system life cycle processes, is a process framework. It describes how AI systems are managed across their life cycle. It does not define an incident taxonomy, a severity scale or a problem management procedure. Security teams therefore have to interpret which of its processes apply once a system is in production, and document that interpretation.

    Treat the standard as a source of process structure, not as a checklist. The mapping in this guide is practitioner guidance: operation, monitoring and maintenance-related processes are the natural home for detection, containment and corrective action, and your own process owners should confirm the fit.

    Two neighbouring items matter. ISO/IEC 42001:2023 is a separate management system standard in the same ISO/IEC JTC 1/SC 42 catalogue, so do not treat it as interchangeable with 5338. ISO/IEC CD 25870, on data elements for reporting AI system incidents, is still under development at committee draft stage. It is not a requirement, but a structured incident record will be easier to align with it later.

    On the regulatory side, conformance to ISO/IEC 5338 should not be assumed to give a presumption of conformity under the EU AI Act. As of November 2026, the Digital Omnibus on AI is in force and moves the high-risk application dates to 2 December 2027 for stand-alone high-risk AI systems and 2 August 2028 for high-risk systems embedded in products. Build timelines on those dates, not on 2 August 2026.

    Separating one-off incidents from recurring problems

    An incident is a single event in which an AI system behaved outside policy or expectation and needs to be contained and resolved. A problem is the underlying cause behind one or more incidents. Keeping the two records separate stops teams from closing the same incident repeatedly without removing what produces it.

    Define explicit triggers for opening a problem record. Useful triggers include the same policy being tripped repeatedly, the same tool or agent appearing in several incidents, or a similar prompt pattern producing similar outcomes. A single high-severity incident with a clearly systemic cause can also justify a problem record immediately.

    There is a tradeoff. A low threshold produces many problem records and dilutes attention. A high threshold lets slow-moving patterns pass unnoticed. Start with a modest threshold, review it quarterly against actual incident data and label the scheme as your own. No external standard currently fixes these categories, and the draft incident-reporting standard should be tracked rather than assumed.

    Retention and evidence quality

    Retention periods should come from the regulatory obligations that apply to your organisation, not from a generic number. Decide in advance how long routine evidence is kept, and whether evidence attached to an open or closed incident is held longer. Make sure retention and data residency settings can be configured to match those decisions.

    Evidence is only useful if it is tamper-resistant, attributable and complete for the scope you claim. Be precise about that scope. Records of individual interactions and tool calls support incident reconstruction for those events. They do not by themselves establish model lineage, dataset provenance or an end-to-end trace of a multi-step agentic workflow, so state those limits in your incident procedures rather than letting auditors infer them.

    Why runtime enforcement changes the evidence

    Reviewing activity after the fact means reconstructing what happened from whatever was captured, often with no record of what the system was permitted to do. Policy enforced at runtime, in the execution path, works differently. Each decision (allow, redact, block, or escalate for human approval) is made before the action completes, and the decision itself becomes the record.

    This helps at two points in the lifecycle. During containment, a policy change takes effect on the next interaction or tool call, so the response to an incident is an enforcement change rather than a request to a team. During investigation, the record shows the policy that applied and the outcome, which is the question auditors ask first.

    Trussed enforces policy in the execution path across models, agents, tools and MCP servers, scoped to individual interactions and tool calls. Enforcement generates continuous audit-ready evidence mapped to frameworks including the EU AI Act and NIST, and the platform deploys self-managed in the customer cloud with configurable retention and data residency. In production it generates 30+ GB of audit evidence per day for regulated enterprises. It does not provide model or dataset provenance, and it does not trace agentic workflows end to end.

    Questions to evaluate any evidence approach

    Whether you build this or buy it, test the approach against the incident lifecycle above.

    • Can you reconstruct a single interaction or tool call, including the policy decision applied, after the incident?
    • Are blocked, redacted and escalated actions recorded, along with human approval or rejection?
    • Can a containment change be applied without redeploying the AI system?
    • Can evidence be linked to incident, problem and corrective action records?
    • Can retention and residency be set to match your regulatory obligations?

    From AI incident to closed problem

    Detect and classify

    Open an incident record with a severity and category.

    Contain and investigate

    Stop further harm, then reconstruct the interaction.

    Root cause and correct

    Link repeat incidents to a problem record and a fix.

    Evidence

    Policy decisions, tool calls and approvals kept for review.

    See what incident-ready evidence looks like

    Walk through how policy enforced in the execution path produces audit-ready evidence for individual interactions and tool calls.

    See How Evidence Is Produced
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.

    Book Demo