AI Agent Failure Modes: A Taxonomy for Risk Teams

Why Model-Level Risk Frameworks Are Insufficient

Most enterprise risk registers were built around single-call model risk: hallucination, bias, unsafe output. Autonomous agents introduce failure surfaces that do not exist in a single inference call. An agent can produce a factually correct reasoning trace and still invoke the wrong tool, act on stale permissions, or continue executing a multi-step task after the underlying goal has been altered by injected content. OWASP's agentic AI guidance treats these as a distinct threat category from standard LLM application risks, separating orchestration and tool-invocation failures from failures originating in the foundation model itself. Risk teams evaluating agent deployments need a taxonomy that reflects this separation, because the controls that mitigate model-level risk (output filtering, fine-tuning, prompt hardening) do not address agent-level or orchestration-level failures.

Layer One: Model-Level Failures

Model-level failures occur within a single inference step and include hallucination, reasoning error, and susceptibility to instructions embedded in the input. NIST's Generative AI Profile identifies confabulation and information integrity risk as relevant categories here. In an agentic context, a model-level failure becomes dangerous not because the text output is wrong, but because that output is subsequently interpreted as an instruction or parameter by the agent's execution layer. Prompt injection, whether direct (crafted user input) or indirect (malicious content retrieved from a tool call or external document), belongs in this layer because it manipulates the model's reasoning state. Indirect injection is particularly relevant to agents, since they routinely process untrusted external content as part of tool execution, and OWASP's LLM Top 10 identifies this as a distinct risk from static single-turn jailbreak attempts.

Layer Two: Agent-Level Failures

Agent-level failures occur when the model's output is translated into action: selecting a tool, constructing parameters, and executing a call. Tool-call errors happen when an agent selects an incorrect tool, passes malformed or incomplete parameters, or invokes an action outside its intended scope. This is a distinct failure category from a bad text response, because the model's reasoning may be sound while the execution step fails. Excessive agency, a category defined in OWASP's LLM Top 10, describes a system granted more functionality, permission, or autonomy than its task requires. Permission misuse is closely related: agent identities are frequently provisioned through service accounts or delegated credentials with broader access than any single task needs, so one misdirected or compromised step can cascade into unauthorized data access or unintended system changes. Risk teams should evaluate agent identity and permission scope as a control domain distinct from model access controls.

Layer Three: Orchestration-Level Failures

Orchestration-level failures emerge from the sequencing and coordination of multiple steps or multiple agents, not from any single model call or tool invocation. Unbounded action chains occur when an agent lacks termination conditions, step limits, or checkpoints, allowing an early error to compound across subsequent tool calls before any human review occurs. Multi-agent systems introduce additional coordination failures, including conflicting goals between agents and manipulation of inter-agent messages, which require controls distinct from single-agent tool misuse. MITRE ATLAS documents adversarial techniques against deployed systems that map to this layer, providing empirical grounding for treating orchestration failures as a technique category rather than a theoretical concern. Visibility into the full sequence of tool calls and intermediate reasoning steps, not just final outputs, is necessary to detect these failures before they complete.

Failure Layer Comparison

The table below summarizes where each failure originates, what it looks like in practice, and the control category most directly responsible for addressing it.

Mapping Failure Modes to Existing Risk Frameworks

NIST's AI RMF recommends mapping identified AI risks into existing organizational risk registers and control libraries rather than building a parallel AI-only framework. In practice, this means agent identities should be reviewed through standard access governance processes, analogous to service account review, rather than a separate agent-specific process. Excessive agency and permission scope should be documented as part of that same review cycle. Vendor claims about agent safety controls should be evaluated against independent frameworks such as OWASP's agentic AI guidance, MITRE ATLAS, and the NIST AI RMF, rather than accepted as self-certified. Sector-specific disclosure obligations for agent failures are not yet standardized across jurisdictions, so risk teams should confirm current requirements within their own regulatory context rather than assume a uniform baseline.

Runtime Enforcement as a Control Layer

Design-time guidance, such as least-privilege tool permissioning and human checkpoints for high-impact actions, addresses failure modes before deployment. Runtime enforcement addresses them during execution. This includes policy-based gating of tool calls, permission scoping enforced at the point of invocation rather than only at provisioning time, and monitoring of agent action sequences to detect anomalous chains before they complete. OWASP's mitigation guidance explicitly recommends tying human-in-the-loop checkpoints to action risk tier rather than applying them uniformly, since uniform checkpoints on every action tend to be bypassed or ignored in practice. Trussed AI provides runtime governance for enterprise AI agents, including permission enforcement, tool approval workflows, and audit logging designed to give risk teams visibility into agent behavior at the point of execution rather than only after an incident has occurred. This is a control layer that complements, rather than replaces, design-time model and access governance.