Kong AI Rate Limiting Advanced for AI Agents: What Token Limits Miss

What token-based rate limiting does well

Rate limiting restricts how many requests a single source can make in a given period. OWASP lists rate limiting and user quotas among its mitigations for unbounded consumption in LLM applications. Token-based limits apply the same idea to model usage. Instead of counting requests, the gateway counts tokens against a limit within a window, scoped to a consumer, route, or model. When a caller exceeds the limit, the gateway rejects further traffic until the window resets.

This is the control that Kong AI Rate Limiting Advanced provides at the gateway, and it earns its place. It bounds the worst case for a runaway client, protects shared model capacity, and gives platform teams a predictable ceiling per consumer. NIST SP 800-228 recommends rate limiting all API access for all callers, and notes that internal callers matter as much as external traffic. Agents running inside the enterprise are internal callers and should be covered.

Two boundaries are worth stating early. NIST also says rate limits are not quotas: quotas need strict enforcement and are typically used for billing reports. And NIST recommends mitigating unrestricted resource consumption with a combination of controls, including rate limiting, quotas, spending policy controls, and abuse detection. Token limits are one layer in that set, not the whole set.

Agent behaviors that token limits do not constrain

A token budget measures how much text moved through a model. It says nothing about what the agent did with the result. Agents chain tool calls, invoke MCP servers, retry failed steps, and fan out across models. Each of those can happen inside a budget that looks healthy.

OWASP describes this gap in several places. Its Excessive Agency guidance recommends rate limiting to reduce the number of undesirable actions in a given period, which increases the chance of discovering them through monitoring before significant damage occurs. That framing matters: rate limiting there buys time for detection. It does not decide whether an action should have happened. The OWASP Cornucopia card AAI4 describes an agent that can loop or chain external tool calls without rate limits or budget controls, and recommends circuit breakers that detect abnormal call rates or repetitive patterns and pause execution pending human review. It also recommends per-user and per-tenant limits applied independently of agent-level limits, so one session cannot affect others.

The OWASP AI Agent Security Cheat Sheet adds action-level concerns: limits on tool-call rate, detection of data exfiltration through tool calls, and separating decision-making from execution for irreversible operations. None of these is a token count. An agent that sends a small number of high-impact messages, writes to a system it should not touch, or passes sensitive data to a tool consumes very few tokens and is not stopped by a token limit.

Consumption limits compared with policy enforcement on interactions and tool calls

These are different controls with different inputs and different possible decisions. The table summarizes the distinction. Most agentic deployments need both.

• What it measures: Consumption limit: tokens or requests per window. Policy enforcement: the content, target, and context of each interaction or tool call.
• Unit of decision: Consumption limit: the caller, route, or model. Policy enforcement: the individual interaction or tool call.
• Possible decisions: Consumption limit: permit traffic or reject it once the limit is reached. Policy enforcement: allow, redact, block, rate limit, or escalate for human approval.
• Tool and MCP calls: Consumption limit: outside its view unless the calls pass through the same enforcement point. Policy enforcement: evaluated in the execution path.
• Irreversible actions: Consumption limit: no distinction between reversible and irreversible. Policy enforcement: can route an action to human approval before it runs.
• Record produced: Consumption limit: evidence that a limit was reached. Policy enforcement: a record of the decision made on each interaction or call.

Enforcing policy in the execution path

Behavioral policy works at a different level of granularity. Rather than counting what passes, it evaluates each interaction and each tool call as it happens and returns a decision. An allow lets the call proceed. A redact removes sensitive content before it reaches a model or tool. A block stops the call. A rate limit slows a specific pattern, such as repeated calls to one tool. An escalation holds the action until a person approves it, which is the pattern OWASP describes for circuit breakers and irreversible operations.

Trussed enforces policy in the execution path across models, agents, tools, and MCP servers, scoped to individual interactions and tool calls. It is designed to sit alongside gateway-level limits, not replace them. The gateway bounds consumption per consumer, route, or model. Runtime policy enforcement decides what each interaction and tool call is permitted to do. Enforcement happens at runtime, not through review after the fact, so a disallowed action is stopped before it completes.

The same decision point is what produces evidence. When enforcement itself generates the record, the record shows the policy that applied, the decision, and the interaction it applied to. Trussed generates this audit-ready evidence continuously, mapped to frameworks including NIST and the EU AI Act. A rate limit record shows that a threshold was reached. It does not show what an agent was permitted to do or what it did, and that is the question a regulated enterprise has to answer for an auditor.

Evaluating complementary runtime controls

Keep token-based limits where they work. They are simple, predictable, and recommended in both OWASP and NIST guidance. Then define which agent behaviors your enterprise has not intended to allow, and test whether any current control can stop them. If the answer depends on a token count, the control is measuring the wrong thing.

When evaluating a runtime enforcement layer, ask whether it evaluates individual interactions and tool calls, how it covers MCP servers, what decisions it can make, and what evidence it generates. Deployment model matters for regulated workloads too. Trussed supports self-managed deployment in the customer cloud, with configurable retention and data residency. It has run in continuous production in a regulated enterprise for more than 18 months, has held 99.95% uptime on live production traffic, and applies one policy plane across 100+ models.

A practical next step is to take one production agent, list every tool and MCP server it can reach, and mark which actions would be acceptable at any volume and which would not. That list is the starting policy set, and it shows where a token limit stops being the relevant control.