How to Write an AI Agent Threat Model in One Day

Why AI agents require a distinct threat model

AI agents extend the attack surface of a traditional application. A single user input can trigger a multi-step tool-call chain that reaches several downstream systems without human review in between. Persistent memory adds a data store that can be poisoned or exfiltrated across sessions, unlike stateless application state. Agents also act under delegated or impersonated identity, raising spoofing and authorization questions that do not exist in conventional service-to-service architectures.

OWASP's Top 10 for LLM Applications formalizes some of this as a distinct risk category, Excessive Agency, describing harm caused by agents granted more functionality, permissions, or autonomy than their task requires, alongside Prompt Injection as a primary risk. MITRE ATLAS catalogs adversary techniques observed against AI systems in a structure modeled on ATT&CK, and NIST's AI Risk Management Framework organizes risk management into Govern, Map, Measure, and Manage functions.

No single authoritative standard yet exists specifically for AI agent threat modeling. Current practice adapts these general AI and LLM frameworks to agentic systems, which is the approach this guide follows.

Mapping agent-specific trust boundaries

Treat the agent as a set of distinct components rather than a single monolithic system. Trust boundaries exist between the agent and each tool it can invoke, between the agent and the underlying model, between cooperating agents, and between the agent and its memory stores.

Short-term context and long-term or persistent memory should be treated as separate assets, each requiring its own data classification, since a compromised memory store can influence agent behavior across unrelated sessions. Standards work on tool-integration protocols, including Anthropic's Model Context Protocol, formalizes this by documenting that tool servers and hosts operate across explicit trust boundaries requiring authorization and consent before an agent can invoke a tool or access a resource. Using this kind of reference model helps a security team map data flows quickly rather than debating boundary definitions during the exercise itself.

Agent-specific risk categories to enumerate (STRIDE-adapted)

Walk each in-scope component and boundary against this checklist. Capture concrete attack paths, not generic labels.

• Spoofing: identity spoofing of the user, the agent, or a downstream service in a delegated identity chain.
• Tampering: poisoning of persistent memory or manipulation of tool outputs that feed subsequent decisions.
• Repudiation: absence of audit logging for tool invocations, preventing reconstruction of a multi-step action chain.
• Information Disclosure: exfiltration of memory contents or context data through a compromised tool call.
• Denial of Service: resource exhaustion caused by unintended or chained tool calls triggered by a single input.
• Elevation of Privilege: excessive agency, where the agent holds more permissions or autonomy than its task requires.

Assigning ownership and feeding ongoing governance

A one-day threat model is a starting point, not a final artifact. NIST's AI RMF Govern function calls for assigning accountability for AI risk decisions, and that accountability should be reflected in who signs off on the exercise output. Joint guidance from NSA, CISA, and international partners on deploying AI systems securely recommends continuous monitoring throughout the AI system lifecycle, which means the mitigations identified during the exercise should be tracked into an operational governance process rather than left as a static document.

Agent-specific risks such as excessive agency should be entered into the same risk register used for traditional application vulnerabilities, with the same review cadence. Time allocations described here are a practitioner methodology, not a codified standard, and teams should adjust phase lengths based on the complexity of the agent and the number of tool integrations in scope.