Technical Guide
What Is an AI Kill Chain? Mapping Attack Stages for LLM Systems
A structured model for how attackers progress against LLM applications and autonomous agents, and where runtime controls can interrupt an in-progress attack.
The Context Window Replaces the Network as the Attack Surface
The traditional cyber kill chain, developed by Lockheed Martin, describes intrusion as seven sequential stages: reconnaissance, weaponization, delivery, exploitation, installation, command and control, and actions on objectives. That model assumes an attacker moving through a network, exploiting a vulnerability, and installing persistent access.
LLM applications and autonomous agents change the terrain. There is no single official AI kill chain standard, but frameworks such as MITRE ATLAS, the OWASP Top 10 for LLM Applications, and NIST's AI Risk Management Framework consistently describe an equivalent progression that plays out inside the context window and tool-call interface rather than the network layer.
In this model, untrusted documents, retrieved search results, and tool outputs become the delivery mechanism. An adversarial instruction embedded in a document, a support ticket, or an API response can reach the model the same way a malicious attachment reaches an inbox. Because autonomous agents can reason, call tools, and act within a single execution turn, several traditional kill chain stages, including delivery, exploitation, and actions on objectives, can compress into one conversational or task cycle, leaving defenders less time to detect and interrupt an attack in progress.
Mapping Traditional Stages to LLM and Agent Attack Techniques
Security teams can use the following stage model to map attacker progression against LLM systems and identify control points. The labels adapt familiar kill-chain language to techniques that target prompts, tools, memory, and connected servers.
| Stage | What the attacker does |
|---|---|
| Reconnaissance | Probing system prompts, tool schemas, and exposed manifests to learn how the agent is configured and what it can invoke. |
| Payload Crafting | Building adversarial instructions intended for embedding in documents, tickets, search results, or other content the model will later consume. |
| Context Injection | Delivering prompt injection through documents, tools, or retrieved data so untrusted text enters the model's working context. |
| Tool-Call Abuse | Coercing the agent into unauthorized actions by shaping tool calls that escape intended permissions or schemas. |
| Context and Memory Poisoning | Persisting instructions across sessions or memory so later turns continue to follow attacker intent. |
| Exfiltration and Impact | Completing unauthorized data access or downstream system manipulation once the agent acts on poisoned context or abusive tool calls. |
MCP and the Expanding Tool Attack Surface
The Model Context Protocol (MCP), introduced by Anthropic in November 2024 as an open standard, illustrates how the attack surface has shifted. MCP allows AI assistants and agents to connect to external data sources and tools through standardized servers, simplifying integration but also standardizing how external tool descriptions and data are passed directly into the model's context.
Security researchers have flagged connected MCP servers as a potential vector for prompt injection and unauthorized tool invocation, since data or tool manifests originating from an MCP server enter the model's context alongside trusted instructions. This is documented primarily through security research and proof-of-concept findings rather than confirmed enterprise breach reports, but the underlying architecture is consistent with how OWASP's Agentic AI guidance and NIST's Generative AI Profile describe supply-chain and component-integration risk.
Each MCP server, plugin, or tool connector should be treated as an independent trust boundary. Without validation of tool manifests and clear separation between trusted system instructions and externally sourced content, a single compromised or malicious connector can inject instructions that influence agent behavior downstream.
Treat every MCP server and tool connector as its own trust boundary. Manifest integrity, provenance checks, and separation of untrusted content from system instructions are prerequisites, not optional hardening.
Implementation Considerations
- Apply least-privilege, time-bound scoping to agent tool permissions instead of static broad grants.
- Log and retain tool-call sequences and context injections to support forensic reconstruction of an attack's progression.
- Implement anomaly detection on tool invocation frequency and patterns to catch automated or scripted abuse.
- Verify the integrity and provenance of MCP server or tool manifests before permitting agent connections.
- Establish content-provenance tagging so retrieved or external data is distinguishable from trusted system instructions at inference time.
- Map each control back to an existing enterprise risk framework, such as NIST AI RMF's Govern, Map, Measure, and Manage functions, for consistent audit and board reporting.
Where Runtime Controls Interrupt the Kill Chain
Runtime controls are most effective when they sit on the path between model output, inbound context, and live systems. The following control points align to the stages above and give security teams concrete places to interrupt an attack before impact.
-
Per-agent identity
Scoped, session-bound credentials distinct from shared service accounts, so a compromised session cannot inherit broad standing access.
-
Tool-call governance
Allowlisting and schema validation positioned between model output and the execution environment, blocking tool-call abuse before it reaches a live system.
-
Runtime policy enforcement
Real-time inspection of inbound context and outbound tool calls, catching injected instructions and unauthorized invocations at the delivery and exploitation stages.
-
Context segregation
Explicit separation of untrusted external content from system and instruction prompts, reducing the chance retrieved data is interpreted as a command.
-
MCP and tool validation
Verifying manifest integrity and provenance for each connected server before permitting agent access, treating the connection itself as a trust boundary.
Questions to Ask When Evaluating AI Agent Security Controls
Use these questions when comparing platforms or reviewing an internal agent architecture. They map directly to identity, policy, observability, MCP trust, and in-progress interruption.
- Does the platform enforce least-privilege, per-agent identity and dynamically scoped tool permissions?
- Can the solution apply runtime policy enforcement to both inbound context and outbound tool calls?
- What logging and observability does the platform provide across each stage of an agent's task execution?
- How does the vendor validate, sandbox, or monitor third-party MCP servers or tool integrations before and during use?
- Can the platform detect and interrupt in-progress prompt injection or context poisoning prior to tool execution?
Map Your AI Attack Surface Before Attackers Do
Trussed AI provides runtime governance for enterprise AI agents, including agent identity, least-privilege tool permissions, MCP security, and runtime policy enforcement across the stages of an AI kill chain.
Request a Demo