AI Agent Security Evaluation

AI Agent Security Questions for Enterprise Buyers

Enterprise buyers evaluating AI agent platforms should require vendors to answer specific technical questions in five areas: how agent identity is verified per action, whether permissions are scoped to least privilege by default, whether a policy enforcement point mediates tool calls in real time, how the vendor's Model Context Protocol implementation handles consent and authorization, and whether audit logs capture per-call detail sufficient for forensic reconstruction. Questions that stop at policy documents or model-level safety claims do not test runtime security posture.

AI Agent Security Evaluation Checklist

Use these questions to structure vendor conversations before deployment. Each one is designed to surface runtime behavior, not policy language.

  • How is agent identity established and verified for each individual action, and is it distinct from any human user's credentials?
  • Can permissions be scoped per tool call or task, and what is the default permission posture: least privilege or broad access?
  • Is there a policy enforcement point that can block or require approval for tool calls in real time, independent of the model's own output?
  • How does the vendor's Model Context Protocol (or equivalent) implementation handle consent, authorization scoping, and validation of third-party tool and data connections?
  • What audit log detail is captured per tool call, including identity, action, parameters, and authorization decision, and can it export to existing SIEM infrastructure?
  • How are agent credentials provisioned, rotated, and revoked independently of human user account lifecycles?

What Runtime Security Means for AI Agent Platforms

AI agents differ from traditional software in one structural respect: they make autonomous decisions about which tools to call, which data to access, and which actions to take, often without a human approving each step. This shifts the security question away from whether a model produces safe outputs and toward whether the surrounding system can verify, constrain, and record what the agent actually does at runtime. Established frameworks from NIST and OWASP treat this as a distinct risk category. NIST's Zero Trust Architecture guidance (SP 800-207) calls for per-request authentication and continuous verification rather than standing credentials issued once and trusted indefinitely. OWASP's work on agentic AI security identifies "excessive agency" and uncontrolled tool invocation as risks separate from prompt injection or output filtering. Together these sources establish a baseline: enterprise buyers should expect AI agent platforms to demonstrate identity verification, scoped permissions, and intercepting policy enforcement as architectural features, not optional add-ons. A vendor evaluation that relies only on model-level safety claims or general risk commentary will not surface whether these controls exist in the actual runtime path between an agent's decision and its execution of that decision.

Agent Identity and Permission Scoping

A recurring pattern in early AI agent deployments is the use of a single static API key or shared service-account credential to authorize everything an agent does. This is functionally equivalent to giving every employee the same master password and is inconsistent with the least-privilege principle that NIST's Zero Trust Architecture guidance applies to any system, human or automated. Buyers should ask whether agent identity is cryptographically verifiable and scoped to a specific session or task, rather than inherited once at startup and held indefinitely. The distinction matters operationally: if an agent's credential is compromised or its behavior drifts, a per-action identity model limits the blast radius to that session, while a static credential exposes every permission the agent was ever granted.

Permission scoping should be evaluated at the same granularity. Fine-grained scoping means permissions are defined per tool or per action, so an agent authorized to read a calendar is not implicitly authorized to send email. Coarse-grained scoping assigns broad access at the agent level, which is easier for a vendor to implement but harder for a buyer to audit or constrain. Ask whether scopes can be narrowed at deployment time through configuration, or whether narrowing access requires a code change on the vendor's side, since the latter indicates permissions were not designed with least privilege in mind.

Model Context Protocol Security Considerations

The Model Context Protocol (MCP) is an open specification for connecting AI models to external tools and data sources, and it has become a common integration layer for agent platforms. MCP's own documentation separates the architecture into hosts, clients, and servers, each representing a distinct trust boundary that a vendor must secure independently rather than assume is covered by a single access control layer. MCP's specification explicitly names risks that buyers should ask vendors to address directly: the confused deputy problem, where an agent is tricked into using its own legitimate permissions on behalf of an unauthorized request; token passthrough, where credentials intended for one service are forwarded to another without re-validation; and over-permissioned tool access, where a connected tool is granted more capability than the agent's task requires. A vendor supporting MCP should be able to describe how consent is requested and validated at each tool invocation, not only when a tool connection is first established. If a vendor cannot explain how their implementation prevents confused deputy or token passthrough scenarios, that is a gap worth treating as a blocking issue rather than a minor follow-up.

Runtime Governance Practices to Verify During Evaluation

  • Confirm a policy enforcement point exists between the agent's decision and tool execution, so unauthorized calls can be blocked before they run rather than logged after the fact.
  • Require the vendor to demonstrate runtime monitoring that is separate from the agent's own reasoning process, since relying on a model to self-report violations is not independent verification.
  • Ask whether secure defaults are shipped out of the box, consistent with CISA's secure-by-design guidance, rather than requiring the buyer to configure hardening after deployment.
  • Verify that audit trails are tamper-evident and support reconstruction of an agent's full decision chain during a post-incident investigation.
  • Map the vendor's documented capabilities against NIST AI RMF's govern, map, measure, and manage functions to confirm lifecycle accountability, not just point-in-time controls.

Five Evaluation Categories for AI Agent Runtime Security

These categories map directly to the checklist above and give buyers a shorthand for structuring vendor scorecards.

Agent Identity

Per-action verification distinct from human user credentials.

Permission Scoping

Least-privilege defaults versus broad, static access.

Policy Enforcement

Real-time mediation of tool calls before execution.

MCP Security

Consent and authorization validation at each invocation.

Auditability

Structured logs capturing identity, action, and decision.

Frequently Asked Questions

What is runtime governance for AI agents?

Runtime governance refers to controls that operate while an agent is executing, such as identity verification, permission checks, and policy enforcement at the moment of a tool call. It is distinct from pre-deployment testing or model-level safety tuning, which do not constrain what an agent can do once it is running in production.

How is AI agent security different from general LLM security?

LLM security typically addresses risks in model outputs, such as prompt injection or harmful content. AI agent security additionally addresses what the agent does with tool access and permissions, since agents take autonomous actions beyond generating text, including calling external systems and modifying data.

Is MCP support alone sufficient evidence of good agent security?

No. MCP defines authorization and consent patterns, but its specification also notes risks like confused deputy and token passthrough that implementations can still get wrong. Supporting MCP indicates a vendor uses a common protocol, not that the protocol has been implemented securely.

Why does static API key access for agents matter to security evaluation?

Static, broadly scoped credentials mean a compromised or misbehaving agent retains every permission it was ever granted, with no per-action limits. This is inconsistent with least-privilege and Zero Trust principles and should be treated as a significant finding during vendor evaluation.

Apply These Questions Before You Deploy

Runtime governance should be a required evaluation category for any AI agent platform, not an afterthought. Use this checklist to structure vendor conversations around identity, permissions, policy enforcement, and auditability.

Explore Runtime Governance