What Runtime Security Means for AI Agent Platforms
AI agents differ from traditional software in one structural respect: they make autonomous decisions about which tools to call, which data to access, and which actions to take, often without a human approving each step. This shifts the security question away from whether a model produces safe outputs and toward whether the surrounding system can verify, constrain, and record what the agent actually does at runtime. Established frameworks from NIST and OWASP treat this as a distinct risk category. NIST's Zero Trust Architecture guidance (SP 800-207) calls for per-request authentication and continuous verification rather than standing credentials issued once and trusted indefinitely. OWASP's work on agentic AI security identifies "excessive agency" and uncontrolled tool invocation as risks separate from prompt injection or output filtering. Together these sources establish a baseline: enterprise buyers should expect AI agent platforms to demonstrate identity verification, scoped permissions, and intercepting policy enforcement as architectural features, not optional add-ons. A vendor evaluation that relies only on model-level safety claims or general risk commentary will not surface whether these controls exist in the actual runtime path between an agent's decision and its execution of that decision.
Agent Identity and Permission Scoping
A recurring pattern in early AI agent deployments is the use of a single static API key or shared service-account credential to authorize everything an agent does. This is functionally equivalent to giving every employee the same master password and is inconsistent with the least-privilege principle that NIST's Zero Trust Architecture guidance applies to any system, human or automated. Buyers should ask whether agent identity is cryptographically verifiable and scoped to a specific session or task, rather than inherited once at startup and held indefinitely. The distinction matters operationally: if an agent's credential is compromised or its behavior drifts, a per-action identity model limits the blast radius to that session, while a static credential exposes every permission the agent was ever granted.
Permission scoping should be evaluated at the same granularity. Fine-grained scoping means permissions are defined per tool or per action, so an agent authorized to read a calendar is not implicitly authorized to send email. Coarse-grained scoping assigns broad access at the agent level, which is easier for a vendor to implement but harder for a buyer to audit or constrain. Ask whether scopes can be narrowed at deployment time through configuration, or whether narrowing access requires a code change on the vendor's side, since the latter indicates permissions were not designed with least privilege in mind.
Model Context Protocol Security Considerations
The Model Context Protocol (MCP) is an open specification for connecting AI models to external tools and data sources, and it has become a common integration layer for agent platforms. MCP's own documentation separates the architecture into hosts, clients, and servers, each representing a distinct trust boundary that a vendor must secure independently rather than assume is covered by a single access control layer. MCP's specification explicitly names risks that buyers should ask vendors to address directly: the confused deputy problem, where an agent is tricked into using its own legitimate permissions on behalf of an unauthorized request; token passthrough, where credentials intended for one service are forwarded to another without re-validation; and over-permissioned tool access, where a connected tool is granted more capability than the agent's task requires. A vendor supporting MCP should be able to describe how consent is requested and validated at each tool invocation, not only when a tool connection is first established. If a vendor cannot explain how their implementation prevents confused deputy or token passthrough scenarios, that is a gap worth treating as a blocking issue rather than a minor follow-up.
Runtime Governance Practices to Verify During Evaluation
- Confirm a policy enforcement point exists between the agent's decision and tool execution, so unauthorized calls can be blocked before they run rather than logged after the fact.
- Require the vendor to demonstrate runtime monitoring that is separate from the agent's own reasoning process, since relying on a model to self-report violations is not independent verification.
- Ask whether secure defaults are shipped out of the box, consistent with CISA's secure-by-design guidance, rather than requiring the buyer to configure hardening after deployment.
- Verify that audit trails are tamper-evident and support reconstruction of an agent's full decision chain during a post-incident investigation.
- Map the vendor's documented capabilities against NIST AI RMF's govern, map, measure, and manage functions to confirm lifecycle accountability, not just point-in-time controls.