Why an LLM Gateway Sees Tool Call Requests but Not Tool Execution

Two events on two paths

A tool call request and a tool execution are different events, and in most agent architectures they happen in different places.

A tool call request is output from the model. When an application supplies tool definitions, the model returns a structured message naming a function and its arguments instead of, or alongside, plain text. The model does not run anything. Public evaluations of function-calling models describe this split: the provider's API specifies how the model formats its output so it can be parsed as a command, and a separate environment then executes that command. Evaluations of plugin systems describe the same pattern, where the model makes the call and the plugin runs its functionality on its own server and returns a response.

A tool execution is what happens after that message is parsed. In an agent framework, the agent runtime or client application reads the tool call and invokes a local function, a plugin server, or an MCP server. The component that issued the request (the model) and the component that acts on it are separated by at least a process boundary and often a network boundary.

An LLM gateway is placed on the model traffic. That is a defined scope, not a defect. It is the right place to apply controls to prompts and completions. It is simply not the place where tools run.

What a gateway can and cannot observe

On the model path, a gateway can see tool definitions in requests, the tool call name and arguments in responses, and tool results when the agent includes them in a follow-up request. It can apply policy to that content, for example by inspecting arguments the model proposed.

It cannot see whether the proposed call was executed, altered, or dropped by the agent runtime. It cannot see which tool server handled it, what files, records, or network endpoints the tool touched, whether the tool failed, or what side effects occurred. If the result is never sent back as a model request, or is transformed before it is, the gateway never sees it. It cannot constrain any of this, because it is not in that path.

This leaves three architectural gaps. First, a gap between requested and executed: the record shows what the model asked for, not what ran. Second, a gap in enforcement: no policy decision is made at the moment of execution. Third, a gap in evidence: nothing produced on the model path can show an auditor what the tool did.

A related point applies to gateways that offer MCP-related management functions. One publicly documented vulnerability in an AI gateway product allowed MCP server preview endpoints to accept a stdio configuration and spawn the command as a subprocess on the proxy host, with the proxy process's privileges. A gateway that manages tools can itself become an execution surface and should be reviewed as one.

Risks when policy covers only model traffic

The execution side carries risks that never appear in the model payload. Tools are invoked with the agent's privileges and can observe its working state. NIST's draft work on adversarial machine learning notes that a malicious tool can exfiltrate user inputs, intermediate results, and credentials directly to attacker-controlled endpoints. None of that traffic crosses the model path.

The return channel is a second exposure. Tool return values are appended to the agent's context and become input to the next action. A tool that controls its return content can embed adversarial instructions that act as indirect prompt injection, steering later actions toward unauthorized tool calls or privilege escalation. Tool descriptions are also untrusted input: a malicious description enters the context even if the tool is never invoked.

Permissions alone may not catch misuse. NIST material on agentic security describes tool misuse as an agent using valid tools in harmful ways, where attackers influence tool choice or arguments and where harmful sequences can remain within policy and within the right permissions. That argues for checking arguments and sequences, not only whether a tool is allowed.

For regulated teams the evidence consequence is direct. An auditor or regulator asks what the AI system was allowed to do and what it did. Model-path records can answer the first half in part and the second half not at all. NIST's discussion material on agentic AI lists records of tool calls, tool usage, and approval events, preserved in a tamper-resistant manner where appropriate, as audit considerations. Those events occur on the execution side.

Enforcement in the execution path

A zero trust model places a policy enforcement point where communication is actually allowed or denied. Applied to agents, the question is which enforcement point, if any, sits at the tool call itself rather than only on model traffic.

Trussed enforces policy in the execution path across models, agents, tools, and MCP servers. The scope is individual interactions and tool calls. Policy is enforced at runtime, not reviewed after the fact, and a tool call can be allowed, redacted, blocked, rate limited, or escalated for human approval. Enforcement itself generates continuous, audit-ready evidence, mapped to HIPAA, NAIC, NYDFS, the EU AI Act, and NIST. Deployment is self-managed in the customer cloud, with configurable retention and data residency. In production, Trussed has run continuously for 18+ months in a regulated enterprise and holds 100+ models under one policy plane.

Two boundaries are worth stating. This is enforcement at the interaction and tool-call level, not end-to-end tracing of agentic workflows. And a gateway on model traffic still has a role; the point is that its scope ends where execution begins, so the execution path needs its own enforcement and its own evidence.