What Should Happen When a Client Sends a Trade Instruction to an AI Assistant
The instruction should pass through policy enforcement in the execution path before the assistant can act on it. At each decision point (instruction recognition, identity and authority, tool-call authorization, data exposure, and approval) the policy returns one outcome: allow, redact, block, rate limit or escalate for human approval. Each decision and each tool call is recorded as evidence. Prompt instructions and after-the-fact review do not stop an unauthorized or misread action.
Decision points between receipt and action
- 1
1. Instruction recognition
Classify whether the message is a possible trade instruction, a question or general conversation. If ambiguous, the assistant may ask the client to clarify, but it must not call an order-affecting tool. Instruction-pattern detection supplements this step as filtering or alerting only.
- 2
2. Identity and authority
Check the authenticated session, not the message text. If identity is unverified or the sender lacks authority over the account, block the action and respond with a fixed, non-revealing message.
- 3
3. Tool-call authorization
Evaluate each requested call against scope, parameters and risk. Allow read-only calls within scope. Block calls outside scope. Escalate order-affecting calls that reach a defined threshold.
- 4
4. Data exposure
Redact account identifiers, positions or other sensitive fields from model context and responses where the sender is not entitled to them.
- 5
5. Rate limiting
Limit repeated or rapid instruction attempts from one session or client to contain loops, retries and probing.
- 6
6. Human approval
Hold the specific call until an authorized person approves or rejects it, and bind that decision to the call.
Evaluating a runtime enforcement layer
- Is policy enforced at tool-call execution time in the request path, not only in prompts or application-side checks?
- Can it allow, block, rate limit or escalate an individual order-affecting call based on identity, scope and risk threshold?
- Does escalation reach the correct human authority, and is the approval bound to the specific call?
- Are client messages and tool outputs treated as untrusted input, with instruction and data contexts kept separate?
- Does it record each interaction and tool call as evidence, and can that map to your documented controls and procedures?
- Can you set retention and data residency, and can the layer be tested by treating the model as an untrusted user?
Why a trade instruction is a runtime problem
A client message that says "sell half of my position" or "cancel that order" is ordinary text to an AI assistant. Large language models accept data and instruction as a single stream of tokens through one interface, so there is no separation between the content the model acts on and the instruction that controls it. The model alone cannot reliably decide whether a message is a binding instruction, a question about a hypothetical, or content that merely resembles one.
The risk becomes concrete at the tool call. With protocols such as MCP, the model decides which tools to invoke, when, and with what parameters. An order or portfolio tool call is therefore the point where a misread instruction becomes an action. Tool outputs matter too: portfolio or order data returned to the model is untrusted input and can carry instruction-like content.
Three control placements are available. Prompt-level instructions that define the assistant's role, capabilities and limits are a recommended mitigation, but system-level instructions that are not structurally protected can be overridden by injected user content. Application-layer checks add logic around the assistant, but they depend on every integration path implementing them consistently. Policy enforced at tool-call execution time, in the path between the assistant and the downstream system, applies the same decision to every call regardless of how the model arrived at it. OWASP guidance points to execution-time policy checks and human confirmation for privileged operations rather than reliance on prompts. Review after the fact can explain what happened. It cannot prevent it.
Identity, authority and tool-call authorization
Identity and authority must come from systems the model cannot influence. A message saying "I am the account holder" or "my adviser approved this" is content, not evidence. The policy layer should evaluate authenticated identity and the entitlements attached to it, and it should do so on every call rather than once per conversation.
Scoped, least-privilege credentials fit this problem well. Access is bounded to the minimum needed for a designated task, and delegated scopes attenuate so a downstream component never holds more authority than the one that delegated to it. For trade instructions this suggests separating read-only portfolio access from order-affecting calls, so that answering a question about a position never carries the authority to change an order. Separation of duties applies as well: the assistant should not hold both the ability to interpret an instruction and the ability to approve it.
In practice, authorization at the tool call should be able to reason over the caller's identity, the tool, the parameters (instrument, side, quantity, order reference) and the organization's risk thresholds. Where a firm is subject to SEC Rule 15c3-5 as a broker or dealer with market access, the rule requires documented risk management controls and supervisory procedures, including financial risk controls reasonably designed to limit exposure from market access. Whether and how it applies to an AI-mediated channel depends on the firm's market-access status and needs legal and compliance review.
Escalation and human approval
Each organization defines the risk thresholds and tolerances that trigger human review, and the technology must be able to route an escalation to the correct authority. For a trade instruction, a threshold might be built from order size, instrument type, account type, deviation from the client's recent activity, or low confidence in the interpretation. The thresholds are a business and compliance decision. The enforcement layer only needs to apply them consistently.
Three design questions determine whether approval is meaningful. First, what exactly is the approver shown? It should be the concrete call with its parameters, not a summary written by the model. Second, is the approval bound to that call? An approval that can be reused for a changed quantity or a different order is not a control. Third, what happens while waiting? The call should stay held, expire after a defined period, and fail closed. Rejection and timeout are outcomes that should be recorded in the same way as approval.
Human-in-the-loop controls are recommended for privileged operations, but approval fatigue is a real tradeoff. If every call escalates, approvers stop reading. Reserve escalation for calls above threshold and use allow, redact, block and rate limit for the rest.
Evidence for the individual interaction
The test of the evidence is whether a reviewer can reconstruct, for one client message, what the assistant was permitted to do and what it did. The records below are a design proposal for the interaction and tool-call scope, to be confirmed against the firm's own supervisory and recordkeeping requirements. Where Rule 15c3-5 applies, the firm must preserve its supervisory procedures and a written description of its risk management controls, so the evidence should map back to those documented controls.
For each interaction, record the authenticated identity and session, the time, the message as received and as passed to the model, and the policy version in force. For each decision point, record the outcome and the reason. For each tool call, record the tool, the parameters, the authorization result, any redaction applied, and the downstream response. For escalations, record who was asked, who decided, when, and which call the decision applied to.
Evidence should be generated by the enforcement point itself, so it describes what was enforced rather than what an application reports afterward. Retention period and data residency are set by the firm's obligations, so they should be configurable rather than fixed by the tool. Protect the records from alteration, and handle sensitive content in them under the same redaction rules as the live interaction.
Lifecycle of one trade instruction
Message received
Client text that reads like buy, sell or change an order is treated as untrusted input.
Policy decision
Identity, authority and risk are evaluated before any tool is called.
Tool-call outcome
Allow, redact, block, rate limit or escalate for human approval.
Evidence
What was permitted and what occurred is recorded for the interaction.
Specify the controls before the first instruction arrives
Review how policy is enforced on individual interactions and tool calls, and how audit-ready evidence is produced for a regulated enterprise.
See How Evidence Is ProducedHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.